-- Logs begin at Mon 2022-09-19 10:40:17 UTC, end at Mon 2022-09-19 11:44:46 UTC. -- Sep 19 11:08:57.039695 dv-cinder-820994-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 19 11:08:57.047011 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 19 11:08:57.049092 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 19 11:08:57 2022] *** Sep 19 11:08:57.049092 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 19 11:08:57.049092 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 19 11:08:57.049092 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: nodename: dv-cinder-820994-5-iscsi Sep 19 11:08:57.049092 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: machine: x86_64 Sep 19 11:08:57.049092 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: clock source: unix Sep 19 11:08:57.049092 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: pcre jit disabled Sep 19 11:08:57.049092 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: detected number of CPU cores: 4 Sep 19 11:08:57.049589 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: current working directory: / Sep 19 11:08:57.049589 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: detected binary path: /usr/bin/uwsgi-core Sep 19 11:08:57.049589 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: your processes number limit is 47680 Sep 19 11:08:57.049589 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: your memory page size is 4096 bytes Sep 19 11:08:57.049589 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: detected max file descriptor number: 2048 Sep 19 11:08:57.049589 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: lock engine: pthread robust mutexes Sep 19 11:08:57.049589 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: thunder lock: enabled Sep 19 11:08:57.049589 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 19 11:08:57.049589 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 19 11:08:57.090961 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: Python main interpreter initialized at 0x55ef24c569c0 Sep 19 11:08:57.090961 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: python threads support enabled Sep 19 11:08:57.090961 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: your server socket listen backlog is limited to 100 connections Sep 19 11:08:57.090961 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: your mercy for graceful operations on workers is 90 seconds Sep 19 11:08:57.091334 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: mapped 403077 bytes (393 KB) for 2 cores Sep 19 11:08:57.091422 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: *** Operational MODE: preforking *** Sep 19 11:08:57.091691 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: *** uWSGI is running in multiple interpreter mode *** Sep 19 11:08:57.091691 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: spawned uWSGI master process (pid: 81449) Sep 19 11:08:57.091918 dv-cinder-820994-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 19 11:08:57.094534 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: spawned uWSGI worker 1 (pid: 81460, cores: 1) Sep 19 11:08:57.094534 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: spawned uWSGI worker 2 (pid: 81461, cores: 1) Sep 19 11:08:57.094534 dv-cinder-820994-5-iscsi devstack@keystone.service[81449]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 19 11:09:01.416531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 11:09:01.416531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: from cryptography.utils import int_from_bytes Sep 19 11:09:01.417332 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 11:09:01.417332 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: from cryptography.utils import int_from_bytes Sep 19 11:09:01.417757 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 11:09:01.417757 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: from cryptography.utils import int_from_bytes Sep 19 11:09:01.417873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 11:09:01.417873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: from cryptography.utils import int_from_bytes Sep 19 11:09:01.467442 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 19 11:09:01.467619 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 19 11:09:01.467710 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 19 11:09:01.467840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 19 11:09:01.467920 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] command line args: [] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 19 11:09:01.467993 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] command line args: [] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 19 11:09:01.468105 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 19 11:09:01.468221 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 19 11:09:01.468314 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ================================================================================ {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 19 11:09:01.468388 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ================================================================================ {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 19 11:09:01.468528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] admin_token = **** {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.468640 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] admin_token = **** {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.468773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] config_dir = [] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.468889 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] config_dir = [] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.468971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.469078 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.469164 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] config_source = [] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.469263 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] config_source = [] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.469342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] debug = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.469445 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] debug = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.469532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.469793 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.469927 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.470035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.470138 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.470236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.470336 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.470439 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.470541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.470644 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.470747 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] list_limit = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.470843 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] list_limit = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.470943 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] log_config_append = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.471044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] log_config_append = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.471138 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.471244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.471244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] log_dir = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.471431 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] log_dir = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.471533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] log_file = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.471629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] log_file = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.471729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.471846 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.471916 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.471977 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.472101 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.472183 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.472327 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.472419 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.472494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.472613 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.472688 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.472766 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.472905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.472992 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.473149 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.473228 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.473381 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.473501 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.473614 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.473700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.473843 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.473927 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.474073 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.474162 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.474321 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.474405 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.474562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.474643 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.474837 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.474917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.475321 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.81/identity {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.475402 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.81/identity {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.475503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] publish_errors = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.475634 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] publish_errors = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.475733 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.475873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.475990 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.476071 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.476151 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.476269 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.476400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.476498 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.476633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.476712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.476864 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] standard_threads = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.476942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] standard_threads = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.477093 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.477176 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.477332 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.477923 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.478043 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.478123 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.478221 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] use_journal = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.478301 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] use_journal = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.478424 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] use_json = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.478504 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] use_json = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.478577 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] use_stderr = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.478697 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] use_stderr = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.478779 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] use_syslog = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.478898 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] use_syslog = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.478982 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.479061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 11:09:01.479183 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.479264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.479420 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.479498 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.479678 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.479777 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.479946 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.480035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.480200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.480279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.480442 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.480522 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.480693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.480772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.480933 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.481014 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.481184 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.481265 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.481429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.481507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.481680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.481759 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.481913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.481992 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.482156 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.482246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.482392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.482481 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.482639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.482719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.482887 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.482973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.483125 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.483209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.483384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.483470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.483635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.483719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.483904 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.483986 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.484138 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.484213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.484401 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.484542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.484643 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.484724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.484884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.484969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.485133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.485212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.485362 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.485451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.485593 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.485679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.485825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.485909 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.486056 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.486145 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.486305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.486382 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.486549 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.486625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.486778 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.486867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.487006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.487084 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.487252 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.487336 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.487499 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.487588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.487732 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.487874 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.488004 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.488092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.488237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.488329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.488475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.488560 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.488711 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.488789 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.488938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.489024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.489169 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.489247 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.489398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.489477 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.489627 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.489781 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.489858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.489928 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.490099 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.490192 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.490330 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.490421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.490565 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.490644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.490793 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.490872 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.491027 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.491118 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.491261 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.491340 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.491506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.491587 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.491743 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.491928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.492088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.492186 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.492281 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.492375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.492470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.492572 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.492675 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.492772 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.492920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.493009 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.493728 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.493854 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.493931 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.494053 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.494132 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.494204 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.494357 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.494447 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.494592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.494723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.494855 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.494978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.495092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.495175 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.495325 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.495408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.495563 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.495651 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.495816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.495901 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.496088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.496171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.496325 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.496406 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.496573 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.496656 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.496818 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.496945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.497052 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.497141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.497286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.497366 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.497519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.497609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.497767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.497877 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.498012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.498120 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.498252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.498357 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.498499 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.498601 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.498737 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.498825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.498988 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.499070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.499228 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.499307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.499464 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.499550 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.499702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.499797 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.499964 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.500044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.500201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.500280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.500461 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.500542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.500700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.500780 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.500948 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.501026 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.501186 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.501267 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.501423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.501501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.501664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.501742 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.501932 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.502023 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.502121 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.502201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.502354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.502442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.502583 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.502677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.502815 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.502900 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.503047 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.503127 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.503280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.503369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.503509 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.503590 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.503750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.503871 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.504007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.504099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.504257 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.504336 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.504494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.504581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.504748 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.504835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.505008 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.505087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.505248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.505355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.505480 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] auth.external = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.505561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] auth.external = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.505710 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.505797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.506006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.506087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.506238 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.506319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.506470 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] auth.password = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.506559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] auth.password = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.506699 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] auth.token = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.506780 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] auth.token = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.506938 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.507017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.507170 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.507258 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.507416 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.507496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.507645 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.507731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.507919 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.508013 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.508167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.508247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.508395 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.508474 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.508631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] credential.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.508719 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] credential.caching = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.508901 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.508982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.509150 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.509236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.509394 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.509483 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.509636 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.509715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.509877 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.509958 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.510112 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.510201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.510357 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.510438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.510609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.510693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.510844 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.510933 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.511096 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.511176 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.511378 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.511457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.511624 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.511711 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.511908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.511996 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.512152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.512232 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.512407 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.512499 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.512642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] federation.caching = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.512722 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] federation.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.512877 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.512957 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.513121 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.513209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.513368 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.513448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.513597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.513677 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.513840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.513928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.514075 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.514155 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.514326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.514405 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.514561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.514651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.514814 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.514895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.515049 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.515129 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.515293 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.515374 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.515526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] identity.caching = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.515605 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.515786 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.515880 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.516028 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.516106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.516260 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.516349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.516493 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.516574 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.516736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.516825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.516964 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.517044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.517198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.517277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.517444 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.517583 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.517693 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.517774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.517924 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.518004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.518152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.518241 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.518383 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.518463 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.518624 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.518703 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.518870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.518951 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.519115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.519195 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.519370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.519450 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.519614 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.519696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.519906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.519989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.520143 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.520224 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.520380 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.520463 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.520610 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.520689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.520846 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.520925 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.521074 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.521155 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.521309 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.521388 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.521549 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.521629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.521794 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.521875 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.522041 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.522125 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.522272 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.522353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.522518 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.522601 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.522766 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.522847 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.522998 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.523086 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.523243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.523329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.523486 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.523567 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.523717 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.523862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.523977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.524058 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.524207 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.524290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.524468 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.524556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.524704 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.524783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.524946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.525026 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.525183 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.525271 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.525420 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.525500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.525666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.525746 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.525912 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.526022 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.526144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.526262 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.526375 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.526498 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.526733 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.526823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.526984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.527067 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.527222 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.527301 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.527457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.527546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.527694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.527792 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.527951 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.user = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.528033 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.528195 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.528288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.528456 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.528537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.528689 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.528770 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.528937 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.529030 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.529184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.529265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.529431 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.529510 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.529665 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.529753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.529897 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.529978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.530133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.530211 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.530366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.530453 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.530604 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.530683 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.530832 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.530911 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.531078 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.531167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.531323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.531405 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.531569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.531649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.531838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.531933 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.532085 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.532173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.532314 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.532395 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.532561 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.532649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.532796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.532876 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.533031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.533111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.533265 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.533353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.533500 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.533579 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.533741 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.533821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.533988 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.534075 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.534222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.534302 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.534475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.534554 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.534702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.534825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.534948 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.535029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.535186 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.535265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.535421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.535511 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.535656 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.535736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.535922 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.536003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.536162 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.536256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.536393 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.536474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.536627 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.536706 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.536867 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] resource.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.536957 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] resource.caching = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.537113 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.537193 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.537361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.537442 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.537597 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.537686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.537844 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.537924 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.538089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.538172 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.538328 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.538415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.538577 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.538657 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.538811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.538890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.539038 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.539114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.539269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] role.caching = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.539351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] role.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.539514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] role.driver = sql {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.539594 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] role.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.539743 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.540371 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.540533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.540613 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.540781 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.540860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.541029 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.541112 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.541273 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.541352 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.541518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.541597 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.541763 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.541845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.542009 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.542088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.542256 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.542336 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.542489 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.542572 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.542735 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.542813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.542984 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.543067 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.543234 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.543317 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.543484 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.543565 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.543898 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.543984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.544139 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.544222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.544393 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.544474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.544643 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.544723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.544891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.544973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.545138 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.545218 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.545371 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.545459 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.545624 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.545704 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.545873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.545952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.546110 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.546202 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.546341 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.546421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.546571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.546650 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.546802 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.546889 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.547050 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.547129 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.547302 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.547381 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.547544 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.547636 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.547794 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.547876 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.548033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.548112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.548261 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.548352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.548494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] token.caching = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.548574 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] token.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.548729 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.548808 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.548973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.549061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.549207 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.549288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.549478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.549596 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.549729 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.549828 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.549941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.550032 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.550183 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.550271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.550422 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.550520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.550667 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.550754 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.550899 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.550986 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.551135 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.551230 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.551366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.551453 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.551609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.551696 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.551901 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.552002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.552140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.552236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.552378 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.552474 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.552639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.552727 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.552898 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] database.connection = **** {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.552986 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.connection = **** {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.553139 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.553224 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.553384 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.553472 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.553619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.553708 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.553858 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.553946 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.554089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.554177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.554323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.554410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.554555 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.554643 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.554787 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.554874 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.555028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.555116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.555264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.555353 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.555521 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.556219 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.556340 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.556428 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.556578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.556667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.556816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.556905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.557063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.557151 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.557298 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.557388 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.557541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.557629 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.557779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.557873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 11:09:01.557960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 19 11:09:01.558035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81461) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 19 11:09:01.587526 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.587754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.588005 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.588106 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.588265 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.588494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.596384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.596860 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.625003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.625235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.625422 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.625657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.678820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.679212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.727501 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.728241 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.728896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.729134 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.883039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.883268 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.883491 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.883730 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.883844 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.883994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.884082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.884213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.884297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.884475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.884651 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.884870 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.907520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.907718 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.907846 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.907997 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.908101 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.908227 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.908313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.908452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.908539 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.908669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.908755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.908885 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.908969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.909103 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.920663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.920993 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.933136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.933376 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.933469 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.933606 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.933693 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.933835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.933919 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.934058 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.934149 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.934274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.934359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.934491 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.934574 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.934708 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81461) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 11:09:01.952030 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.952242 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 11:09:01.952486 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.952571 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.952700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 11:09:01.952785 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 11:09:01.953041 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.953127 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.953241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 11:09:01.953325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.953518 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.953613 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.953784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.953867 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.954046 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.954307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.963932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.964215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.964478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.964564 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.964757 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.964842 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.965029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.965316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.971669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:01.972379 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:01.974572 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:01.974862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.975149 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.975233 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:01.975438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.975523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.975730 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.975842 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.976018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.976109 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.976292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.976377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.976570 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.976658 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.976847 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.976931 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.977124 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.977216 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.977408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.977492 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.977696 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.977970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:01.999930 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.000437 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.001390 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.001685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.001894 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.002183 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.006350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.006637 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.006923 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.007007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.007268 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.007558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.013823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.014112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.014401 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.014693 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.019680 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.020006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.020282 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.020368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.020582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.020667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.020863 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.020949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.021140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.021225 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.021414 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.021510 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.021688 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.021774 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.021966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.022243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.037671 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.037955 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.038365 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.038650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.041829 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.042485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.045087 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.045390 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.045664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.045965 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.049582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.050127 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.052722 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.053204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.055922 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.056229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.056427 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.056514 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.056740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.056826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.057034 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.057129 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.057328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.057413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.057610 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.057892 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.073279 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.073570 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.073844 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.073972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.074115 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.074266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.074540 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.074810 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.081115 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.081908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.082625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.083448 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.084141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.084422 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.084999 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.085294 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.087658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.087969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.088256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.088553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.088636 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.088909 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.089005 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.089205 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.089494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.089895 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.100015 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.100303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.100586 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.100870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.100980 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.101262 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.101348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.101546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.101631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.101836 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.101920 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.102219 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.102541 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.102825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.117230 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.118231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.120172 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.121227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.123049 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.124114 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.124524 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.125557 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.125994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.126281 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.126678 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.127033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.127322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.127723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.133058 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.134646 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.135029 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.136126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.137935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.138334 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.138622 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.138904 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.138988 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.139384 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.139674 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.139975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.146685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.146980 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.147257 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.147756 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.148072 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.148358 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.154669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.154958 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.155247 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.155531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.155708 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81461) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 11:09:02.155834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.156027 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.156111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.156314 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.156401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.156591 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.156674 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.156865 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.156961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.157141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.157226 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.157417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.157551 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.157708 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.157819 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.157997 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.158081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.158270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.158357 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.158553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.158831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.159125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.159407 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81461) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 11:09:02.191945 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ef24c569c0 pid: 81460 (default app) Sep 19 11:09:02.193249 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ef24c569c0 pid: 81461 (default app) Sep 19 11:09:02.195144 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8babab-68d6-4ce2-a95d-47afa44bc4ce None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:02.195430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8babab-68d6-4ce2-a95d-47afa44bc4ce None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:02.195704 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8babab-68d6-4ce2-a95d-47afa44bc4ce None None] PATH_INFO: `/v3` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:02.200356 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1/1] 10.222.0.81 () {50 vars in 791 bytes} [Mon Sep 19 11:09:02 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:08.955113 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf57ee5-a4f9-4b4f-a1cc-cbb6b7009376 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:08.955619 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf57ee5-a4f9-4b4f-a1cc-cbb6b7009376 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:08.955619 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf57ee5-a4f9-4b4f-a1cc-cbb6b7009376 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:08.959814 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1/2] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:09:08.965750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51f58e-93a5-44b8-9afe-9a9eebbb203f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:08.966335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51f58e-93a5-44b8-9afe-9a9eebbb203f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:08.966335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51f58e-93a5-44b8-9afe-9a9eebbb203f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:09.042120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: INFO dbcounter [None req-0e51f58e-93a5-44b8-9afe-9a9eebbb203f None None] Registered counter for database keystone Sep 19 11:09:09.061429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG oslo_db.sqlalchemy.engines [None req-0e51f58e-93a5-44b8-9afe-9a9eebbb203f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81460) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 11:09:09.067560 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writer thread running {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 11:09:09.246196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG passlib.handlers.bcrypt [None req-0e51f58e-93a5-44b8-9afe-9a9eebbb203f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81460) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 19 11:09:09.246513 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG passlib.handlers.bcrypt [None req-0e51f58e-93a5-44b8-9afe-9a9eebbb203f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81460) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 19 11:09:09.308498 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-0e51f58e-93a5-44b8-9afe-9a9eebbb203f None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:09.310133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0e51f58e-93a5-44b8-9afe-9a9eebbb203f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:09.437068 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 2/3] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 19 11:09:09.893589 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf57ee5-a4f9-4b4f-a1cc-cbb6b7009376 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:09.963275 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: INFO dbcounter [None req-cdf57ee5-a4f9-4b4f-a1cc-cbb6b7009376 None None] Registered counter for database keystone Sep 19 11:09:09.984368 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG oslo_db.sqlalchemy.engines [None req-cdf57ee5-a4f9-4b4f-a1cc-cbb6b7009376 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81461) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 11:09:09.990577 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writer thread running {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 11:09:10.114845 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf57ee5-a4f9-4b4f-a1cc-cbb6b7009376 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:10.119663 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd1a00-e949-4f13-bc0f-f4d30f515476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:10.120323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd1a00-e949-4f13-bc0f-f4d30f515476 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:10.120603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd1a00-e949-4f13-bc0f-f4d30f515476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:10.120875 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd1a00-e949-4f13-bc0f-f4d30f515476 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:10.129137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fd1a00-e949-4f13-bc0f-f4d30f515476 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:10.180328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a3fd1a00-e949-4f13-bc0f-f4d30f515476 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 19 11:09:10.180673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fd1a00-e949-4f13-bc0f-f4d30f515476 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:10.190671 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 2/4] 10.222.0.81 () {58 vars in 1324 bytes} [Mon Sep 19 11:09:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 11:09:10.196621 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51f58e-93a5-44b8-9afe-9a9eebbb203f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:10.213828 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51f58e-93a5-44b8-9afe-9a9eebbb203f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:10.218225 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5e97e-5214-4ff5-812f-b119f5642c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:10.219328 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5e97e-5214-4ff5-812f-b119f5642c3e admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:10.219612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5e97e-5214-4ff5-812f-b119f5642c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:10.219922 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5e97e-5214-4ff5-812f-b119f5642c3e admin admin] PATH_INFO: `/v3/projects/5d5c1387f77e4df7ac9b4d83d404bc68` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:10.221508 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd5e97e-5214-4ff5-812f-b119f5642c3e admin admin] RBAC: Authorizing `identity:get_project(project_id=5d5c1387f77e4df7ac9b4d83d404bc68)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:10.274350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2fd5e97e-5214-4ff5-812f-b119f5642c3e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 19 11:09:10.274695 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd5e97e-5214-4ff5-812f-b119f5642c3e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:10.279302 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 3/5] 10.222.0.81 () {56 vars in 1207 bytes} [Mon Sep 19 11:09:10 2022] GET /identity/v3/projects/5d5c1387f77e4df7ac9b4d83d404bc68 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:12.410824 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855716e-4b4e-4572-877b-cf6a7fe8dbcc None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:12.411413 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855716e-4b4e-4572-877b-cf6a7fe8dbcc None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:12.411413 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855716e-4b4e-4572-877b-cf6a7fe8dbcc None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:12.415433 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 3/6] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:12.421374 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d3514-ff49-4ef3-b9ec-8e2dd7023987 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:12.421796 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d3514-ff49-4ef3-b9ec-8e2dd7023987 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:12.421925 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d3514-ff49-4ef3-b9ec-8e2dd7023987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:12.442789 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e30d3514-ff49-4ef3-b9ec-8e2dd7023987 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:12.444333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e30d3514-ff49-4ef3-b9ec-8e2dd7023987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:12.452472 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 4/7] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 19 11:09:12.713993 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855716e-4b4e-4572-877b-cf6a7fe8dbcc None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:12.726007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855716e-4b4e-4572-877b-cf6a7fe8dbcc None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:12.730716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e1ebe5-1985-4ab8-9871-bf29eaff848b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:12.731310 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1ebe5-1985-4ab8-9871-bf29eaff848b admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:12.731586 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1ebe5-1985-4ab8-9871-bf29eaff848b admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:12.731886 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1ebe5-1985-4ab8-9871-bf29eaff848b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:12.739321 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e1ebe5-1985-4ab8-9871-bf29eaff848b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:12.741098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e1ebe5-1985-4ab8-9871-bf29eaff848b admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:12.750334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 4/8] 10.222.0.81 () {58 vars in 1324 bytes} [Mon Sep 19 11:09:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:09:12.755447 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d3514-ff49-4ef3-b9ec-8e2dd7023987 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:12.764724 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d3514-ff49-4ef3-b9ec-8e2dd7023987 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:12.769068 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc1e50-30a4-4654-8c96-ddabd17faad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:12.769689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc1e50-30a4-4654-8c96-ddabd17faad8 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:12.769973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc1e50-30a4-4654-8c96-ddabd17faad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:12.770249 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc1e50-30a4-4654-8c96-ddabd17faad8 admin admin] PATH_INFO: `/v3/users/dd316848081f4ae2ab8647785189b3f8` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:12.771559 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc1e50-30a4-4654-8c96-ddabd17faad8 admin admin] RBAC: Authorizing `identity:get_user(user_id=dd316848081f4ae2ab8647785189b3f8)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:12.773436 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc1e50-30a4-4654-8c96-ddabd17faad8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:12.774958 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 5/9] 10.222.0.81 () {56 vars in 1198 bytes} [Mon Sep 19 11:09:12 2022] GET /identity/v3/users/dd316848081f4ae2ab8647785189b3f8 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:15.005976 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6b51e-9536-4b5d-aa51-053b8c2ea27a None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:15.006488 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6b51e-9536-4b5d-aa51-053b8c2ea27a None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:15.006488 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6b51e-9536-4b5d-aa51-053b8c2ea27a None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:15.007784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 5/10] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:15.013574 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed78312-86a4-459b-b0e6-35f6833e7604 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:15.013985 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed78312-86a4-459b-b0e6-35f6833e7604 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:15.014297 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed78312-86a4-459b-b0e6-35f6833e7604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:15.015747 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a85bc-e420-422a-aa98-0d9f8e080f2a None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:15.016059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a85bc-e420-422a-aa98-0d9f8e080f2a None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:15.016333 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a85bc-e420-422a-aa98-0d9f8e080f2a None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:15.017541 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 6/11] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:09:15.022460 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda8acb-4253-4ade-b47a-af53f1b305fb None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:15.022744 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda8acb-4253-4ade-b47a-af53f1b305fb None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:15.023020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda8acb-4253-4ade-b47a-af53f1b305fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:15.033419 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-aed78312-86a4-459b-b0e6-35f6833e7604 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:15.034613 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-aed78312-86a4-459b-b0e6-35f6833e7604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:15.065100 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG passlib.handlers.bcrypt [None req-ffda8acb-4253-4ade-b47a-af53f1b305fb None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81461) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 19 11:09:15.065501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG passlib.handlers.bcrypt [None req-ffda8acb-4253-4ade-b47a-af53f1b305fb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81461) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 19 11:09:15.115016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-ffda8acb-4253-4ade-b47a-af53f1b305fb None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:15.116681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ffda8acb-4253-4ade-b47a-af53f1b305fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:15.120083 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 6/12] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:15.216626 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 7/13] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:09:15.392960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed78312-86a4-459b-b0e6-35f6833e7604 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:15.419595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:15.480711 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda8acb-4253-4ade-b47a-af53f1b305fb None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:15.505829 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed78312-86a4-459b-b0e6-35f6833e7604 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:15.540524 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e545b-d267-44c1-a854-678b036be09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:15.541187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e545b-d267-44c1-a854-678b036be09d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:15.541460 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e545b-d267-44c1-a854-678b036be09d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:15.541729 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e545b-d267-44c1-a854-678b036be09d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:15.553126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e545b-d267-44c1-a854-678b036be09d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:15.561888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-7b1e545b-d267-44c1-a854-678b036be09d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:15.561888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:15.561888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:15.561888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:15.561888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:15.561888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:15.561888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:15.561888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:15.561888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:15.561888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:15.561888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:15.561888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:15.561888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:15.561888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:15.561888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:15.561888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:15.562826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:15.562826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:15.562826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:15.562826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:15.562826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:15.562826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:15.562826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:15.562826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:15.562826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:15.562826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:15.562826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:15.562826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:15.562826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:15.562826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:15.562826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:15.562826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:15.563663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:15.563663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:15.563663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:15.563663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:15.563663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:15.563663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:15.563663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:15.563663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:15.563663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:15.563663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:15.563663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:15.563663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:09:15.563663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 7/14] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:09:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:15.568939 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e545b-d267-44c1-a854-678b036be09d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:15.583215 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda8acb-4253-4ade-b47a-af53f1b305fb None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:15.585527 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:15.622446 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a1dd2-e264-423a-bddc-d3362eb52317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:15.623321 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a1dd2-e264-423a-bddc-d3362eb52317 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:15.623606 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a1dd2-e264-423a-bddc-d3362eb52317 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:15.623914 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a1dd2-e264-423a-bddc-d3362eb52317 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:15.652739 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:15.671458 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e545b-d267-44c1-a854-678b036be09d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:15.693196 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a1dd2-e264-423a-bddc-d3362eb52317 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:15.694757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a1dd2-e264-423a-bddc-d3362eb52317 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:15.705837 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117505a-046b-4cb0-967e-e02aa4ba4834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:15.706445 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117505a-046b-4cb0-967e-e02aa4ba4834 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:15.706729 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117505a-046b-4cb0-967e-e02aa4ba4834 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:15.707007 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117505a-046b-4cb0-967e-e02aa4ba4834 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:15.707682 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117505a-046b-4cb0-967e-e02aa4ba4834 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:15.708019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117505a-046b-4cb0-967e-e02aa4ba4834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:15.709412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117505a-046b-4cb0-967e-e02aa4ba4834 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:15.721462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 8/15] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:15.727114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117505a-046b-4cb0-967e-e02aa4ba4834 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:15.793359 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 8/16] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 11:09:15.798865 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a1dd2-e264-423a-bddc-d3362eb52317 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:15.801788 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:15.832561 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117505a-046b-4cb0-967e-e02aa4ba4834 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:15.865461 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b8ff7a-a422-4a43-b71f-00acee803f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:15.866084 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8ff7a-a422-4a43-b71f-00acee803f6a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:15.866366 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8ff7a-a422-4a43-b71f-00acee803f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:15.866636 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8ff7a-a422-4a43-b71f-00acee803f6a None admin] PATH_INFO: `/v3/users/dd316848081f4ae2ab8647785189b3f8` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:15.867791 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b8ff7a-a422-4a43-b71f-00acee803f6a None admin] RBAC: Authorizing `identity:get_user(user_id=dd316848081f4ae2ab8647785189b3f8)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:15.869757 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b8ff7a-a422-4a43-b71f-00acee803f6a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:15.871320 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 9/17] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:15 2022] GET /identity/v3/users/dd316848081f4ae2ab8647785189b3f8 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:15.876708 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b8ff7a-a422-4a43-b71f-00acee803f6a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:15.892481 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:15.900125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a1dd2-e264-423a-bddc-d3362eb52317 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:15.934045 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f5291-a11b-4c2f-a5c1-cb9054405c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:15.934822 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f5291-a11b-4c2f-a5c1-cb9054405c32 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:15.935100 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f5291-a11b-4c2f-a5c1-cb9054405c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:15.935369 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f5291-a11b-4c2f-a5c1-cb9054405c32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:15.954145 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f5291-a11b-4c2f-a5c1-cb9054405c32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:15.955614 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f5291-a11b-4c2f-a5c1-cb9054405c32 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:15.973650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-a66f5291-a11b-4c2f-a5c1-cb9054405c32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:15.973650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:15.973650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:15.973650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:15.973650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:15.973650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:15.973650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:15.973650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:15.973650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:15.973650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:15.973650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:15.973650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:15.973650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:09:15.973650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:09:15.973650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:09:15.973650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:09:15.974723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:15.974723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:15.974723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:15.974723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:15.974723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:15.974723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:15.974723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:15.974723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:15.974723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:15.974723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:15.974723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:15.974723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:15.974723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:15.974723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:15.974723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:15.974723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:15.974723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:09:15.975712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:09:15.975712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:15.975712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:09:15.975712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 9/18] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:09:15.979562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b8ff7a-a422-4a43-b71f-00acee803f6a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:15.984053 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f5291-a11b-4c2f-a5c1-cb9054405c32 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:16.019435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340653e-955b-448c-9592-b62c6219e5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:16.020114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340653e-955b-448c-9592-b62c6219e5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:16.020395 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340653e-955b-448c-9592-b62c6219e5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:16.020669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340653e-955b-448c-9592-b62c6219e5e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:16.021798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340653e-955b-448c-9592-b62c6219e5e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:16.023267 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340653e-955b-448c-9592-b62c6219e5e1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:16.024903 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 10/19] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:16.030715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340653e-955b-448c-9592-b62c6219e5e1 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:16.037459 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:16.082900 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f5291-a11b-4c2f-a5c1-cb9054405c32 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:16.111401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:16.116767 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133a56b-512e-4342-b8c2-a034f9b01c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:16.117372 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133a56b-512e-4342-b8c2-a034f9b01c96 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:16.117654 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133a56b-512e-4342-b8c2-a034f9b01c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:16.117929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133a56b-512e-4342-b8c2-a034f9b01c96 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:16.118573 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133a56b-512e-4342-b8c2-a034f9b01c96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:16.119260 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133a56b-512e-4342-b8c2-a034f9b01c96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:16.120357 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133a56b-512e-4342-b8c2-a034f9b01c96 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:16.131007 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340653e-955b-448c-9592-b62c6219e5e1 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:16.137841 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 10/20] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:16.167088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d334f8-9932-4004-a852-958b443de043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:16.167719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d334f8-9932-4004-a852-958b443de043 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:16.168031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d334f8-9932-4004-a852-958b443de043 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:16.168303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d334f8-9932-4004-a852-958b443de043 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:16.169053 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d334f8-9932-4004-a852-958b443de043 None admin] RBAC: Adding query filter params (role.id=753f8b35d5f0458d8f62d0ad6e28033f, scope.domain.id=default, user.id=dd316848081f4ae2ab8647785189b3f8) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:16.169361 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d334f8-9932-4004-a852-958b443de043 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:16.170846 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d334f8-9932-4004-a852-958b443de043 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:16.178546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 11/21] 10.222.0.81 () {56 vars in 1322 bytes} [Mon Sep 19 11:09:16 2022] GET /identity/v3/role_assignments?user.id=dd316848081f4ae2ab8647785189b3f8&scope.domain.id=default&role.id=753f8b35d5f0458d8f62d0ad6e28033f => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:18.693314 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7eb45c-7a8c-4775-afde-be770b6ea720 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:18.693833 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7eb45c-7a8c-4775-afde-be770b6ea720 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:18.693910 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7eb45c-7a8c-4775-afde-be770b6ea720 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:18.695500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 11/22] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:18.700952 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f424fbe-76ed-4c8b-b1f5-9b7e0544784b None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:18.701351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f424fbe-76ed-4c8b-b1f5-9b7e0544784b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:18.701512 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f424fbe-76ed-4c8b-b1f5-9b7e0544784b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:18.729373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-7f424fbe-76ed-4c8b-b1f5-9b7e0544784b None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:18.730614 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7f424fbe-76ed-4c8b-b1f5-9b7e0544784b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:18.754260 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b903e8f-87c5-4f15-b4b3-8a4f8afde7b3 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:18.754551 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b903e8f-87c5-4f15-b4b3-8a4f8afde7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:18.754798 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b903e8f-87c5-4f15-b4b3-8a4f8afde7b3 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:18.756294 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 12/23] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:18.761143 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cd0e9-0b4e-4afa-865d-55d92ea42eca None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:18.761709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cd0e9-0b4e-4afa-865d-55d92ea42eca None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:18.762248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cd0e9-0b4e-4afa-865d-55d92ea42eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:18.800984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-da5cd0e9-0b4e-4afa-865d-55d92ea42eca None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:18.802618 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-da5cd0e9-0b4e-4afa-865d-55d92ea42eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:18.865251 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 12/24] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:18.910039 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 13/25] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:09:19.149590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998987e-5913-4633-bac1-8f15ac229401 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:19.150298 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998987e-5913-4633-bac1-8f15ac229401 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:19.151329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998987e-5913-4633-bac1-8f15ac229401 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:19.152926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 13/26] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:19.158531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ac5de-4aaa-4416-820e-4d1f56c66fe4 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:19.158818 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ac5de-4aaa-4416-820e-4d1f56c66fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:19.159095 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ac5de-4aaa-4416-820e-4d1f56c66fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:19.188759 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-035ac5de-4aaa-4416-820e-4d1f56c66fe4 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:19.189954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-035ac5de-4aaa-4416-820e-4d1f56c66fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:19.249865 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:19.265219 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998987e-5913-4633-bac1-8f15ac229401 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:19.301352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 14/27] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:19.386314 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:19.421591 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998987e-5913-4633-bac1-8f15ac229401 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:19.466646 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1c3bc-b29d-4b30-bd0e-1dbe0aae1a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:19.467287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1c3bc-b29d-4b30-bd0e-1dbe0aae1a4f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:19.467569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1c3bc-b29d-4b30-bd0e-1dbe0aae1a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:19.467868 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1c3bc-b29d-4b30-bd0e-1dbe0aae1a4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:19.484793 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1c3bc-b29d-4b30-bd0e-1dbe0aae1a4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:19.489693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-f4d1c3bc-b29d-4b30-bd0e-1dbe0aae1a4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:19.489693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:19.489693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:19.489693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:19.489693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:19.489693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:19.489693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:19.489693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:19.489693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:19.489693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:19.489693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:19.489693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:19.489693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:19.489693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:19.489693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:19.489693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:19.490528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:19.490528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:19.490528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:19.490528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:19.490528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:19.490528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:19.490528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:19.490528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:19.490528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:19.490528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:19.490528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:19.490528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:19.490528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:19.490528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:19.490528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:19.490528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:19.491919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:19.491919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:19.491919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:19.491919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:19.491919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:19.491919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:19.491919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:19.491919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:19.491919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:19.491919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:19.491919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:19.491919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:09:19.491919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 14/28] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:09:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:19.496699 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ac5de-4aaa-4416-820e-4d1f56c66fe4 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:19.508732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1c3bc-b29d-4b30-bd0e-1dbe0aae1a4f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:19.624525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ac5de-4aaa-4416-820e-4d1f56c66fe4 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:19.663366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb20c35-0919-4ba1-9eb8-5de1cb290211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:19.664377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb20c35-0919-4ba1-9eb8-5de1cb290211 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:19.664858 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb20c35-0919-4ba1-9eb8-5de1cb290211 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:19.665326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb20c35-0919-4ba1-9eb8-5de1cb290211 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:19.671814 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb20c35-0919-4ba1-9eb8-5de1cb290211 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:19.672390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb20c35-0919-4ba1-9eb8-5de1cb290211 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:19.679297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb20c35-0919-4ba1-9eb8-5de1cb290211 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:19.706192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 15/29] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:19.711675 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb20c35-0919-4ba1-9eb8-5de1cb290211 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:19.721310 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1c3bc-b29d-4b30-bd0e-1dbe0aae1a4f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:19.785461 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:19.792586 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288199e-3f1b-4ab3-a562-dbf8c70b7f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:19.793506 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288199e-3f1b-4ab3-a562-dbf8c70b7f12 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:19.793931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288199e-3f1b-4ab3-a562-dbf8c70b7f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:19.794350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288199e-3f1b-4ab3-a562-dbf8c70b7f12 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:19.795221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4288199e-3f1b-4ab3-a562-dbf8c70b7f12 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:19.796880 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4288199e-3f1b-4ab3-a562-dbf8c70b7f12 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:19.807792 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:19.866475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb20c35-0919-4ba1-9eb8-5de1cb290211 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:19.906720 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf2e6f-30e8-425e-8ef1-9601b86470d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:19.907684 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf2e6f-30e8-425e-8ef1-9601b86470d3 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:19.908176 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf2e6f-30e8-425e-8ef1-9601b86470d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:19.908608 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf2e6f-30e8-425e-8ef1-9601b86470d3 None admin] PATH_INFO: `/v3/users/dd316848081f4ae2ab8647785189b3f8` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:19.910240 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf2e6f-30e8-425e-8ef1-9601b86470d3 None admin] RBAC: Authorizing `identity:get_user(user_id=dd316848081f4ae2ab8647785189b3f8)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:19.911965 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf2e6f-30e8-425e-8ef1-9601b86470d3 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:19.914087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 16/30] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:19 2022] GET /identity/v3/users/dd316848081f4ae2ab8647785189b3f8 => generated 247 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:19.915007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf2e6f-30e8-425e-8ef1-9601b86470d3 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:19.945474 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 15/31] 10.222.0.81 () {60 vars in 1131 bytes} [Mon Sep 19 11:09:19 2022] POST /identity/v3/roles => generated 219 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:09:19.947174 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dc722-d41c-4ebd-ad89-32c2279e2077 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:19.947588 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dc722-d41c-4ebd-ad89-32c2279e2077 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:19.947986 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dc722-d41c-4ebd-ad89-32c2279e2077 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:19.949713 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 16/32] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:19.950682 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dc722-d41c-4ebd-ad89-32c2279e2077 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:20.031037 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf2e6f-30e8-425e-8ef1-9601b86470d3 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:20.040880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:20.077336 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2483a1-cfbd-4cfc-9dd0-c2caa2bd80a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:20.078154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2483a1-cfbd-4cfc-9dd0-c2caa2bd80a1 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:20.078667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2483a1-cfbd-4cfc-9dd0-c2caa2bd80a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:20.079011 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2483a1-cfbd-4cfc-9dd0-c2caa2bd80a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:20.091387 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dc722-d41c-4ebd-ad89-32c2279e2077 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:20.091589 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2483a1-cfbd-4cfc-9dd0-c2caa2bd80a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:20.093147 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2483a1-cfbd-4cfc-9dd0-c2caa2bd80a1 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:20.110237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-fb2483a1-cfbd-4cfc-9dd0-c2caa2bd80a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:20.110237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:20.110237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:20.110237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:20.110237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:20.110237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:20.110237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:20.110237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:20.110237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:20.110237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:20.110237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:20.110237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:20.110237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:09:20.110237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:09:20.110237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:09:20.110237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:09:20.111329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:20.111329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:20.111329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:20.111329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:20.111329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:20.111329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:20.111329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:20.111329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:20.111329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:20.111329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:20.111329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:20.111329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:20.111329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:20.111329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:20.111329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:20.111329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:20.111329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:09:20.112268 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:09:20.112268 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:09:20.112268 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:09:20.118750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 17/33] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:20.121032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3deed6-1b5d-49db-a3f5-a357adb5257c None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:20.121522 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3deed6-1b5d-49db-a3f5-a357adb5257c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:20.121987 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3deed6-1b5d-49db-a3f5-a357adb5257c None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:20.124025 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 18/34] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:20.125419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515ef6a-6218-4437-98ba-1cda81cb4f4b None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:20.125843 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515ef6a-6218-4437-98ba-1cda81cb4f4b None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:20.126298 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515ef6a-6218-4437-98ba-1cda81cb4f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:20.133404 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa59690-b24a-4def-b238-485bc797a0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:20.134057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa59690-b24a-4def-b238-485bc797a0bc None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:20.134341 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa59690-b24a-4def-b238-485bc797a0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:20.134634 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa59690-b24a-4def-b238-485bc797a0bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:20.136975 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa59690-b24a-4def-b238-485bc797a0bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:20.138574 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa59690-b24a-4def-b238-485bc797a0bc None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:20.140373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 17/35] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:20.141295 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa59690-b24a-4def-b238-485bc797a0bc None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:20.148663 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-9515ef6a-6218-4437-98ba-1cda81cb4f4b None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:20.150719 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-9515ef6a-6218-4437-98ba-1cda81cb4f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:20.220050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:20.251902 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 19/36] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:20.253655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a1297-f7eb-44ba-a1d8-1cf5a8453d21 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:20.253945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a1297-f7eb-44ba-a1d8-1cf5a8453d21 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:20.254220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a1297-f7eb-44ba-a1d8-1cf5a8453d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:20.270714 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa59690-b24a-4def-b238-485bc797a0bc None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:20.278038 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-0b6a1297-f7eb-44ba-a1d8-1cf5a8453d21 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:20.280028 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-0b6a1297-f7eb-44ba-a1d8-1cf5a8453d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:20.315024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc4b72-05ef-476e-9c35-02fce5823787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:20.315798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc4b72-05ef-476e-9c35-02fce5823787 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:20.316077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc4b72-05ef-476e-9c35-02fce5823787 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:20.316358 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc4b72-05ef-476e-9c35-02fce5823787 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:20.317013 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc4b72-05ef-476e-9c35-02fce5823787 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:20.317581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc4b72-05ef-476e-9c35-02fce5823787 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:20.319686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc4b72-05ef-476e-9c35-02fce5823787 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:20.348470 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 18/37] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:20.348470 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc4b72-05ef-476e-9c35-02fce5823787 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:20.374045 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 20/38] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:20.375983 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a1297-f7eb-44ba-a1d8-1cf5a8453d21 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:20.416848 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:20.522701 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc4b72-05ef-476e-9c35-02fce5823787 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:20.545600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a1297-f7eb-44ba-a1d8-1cf5a8453d21 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:20.548331 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:20.560507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfd895-a961-4e70-91c6-be31d5397ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:20.561175 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfd895-a961-4e70-91c6-be31d5397ed9 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:20.561458 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfd895-a961-4e70-91c6-be31d5397ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:20.561736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfd895-a961-4e70-91c6-be31d5397ed9 None admin] PATH_INFO: `/v3/domains/default/users/dd316848081f4ae2ab8647785189b3f8/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:20.563786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfd895-a961-4e70-91c6-be31d5397ed9 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=dd316848081f4ae2ab8647785189b3f8, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:20.565260 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfd895-a961-4e70-91c6-be31d5397ed9 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:20.580431 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e8f63-1597-494c-a1b6-c4881d1be4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:20.580629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 19/39] 10.222.0.81 () {58 vars in 1361 bytes} [Mon Sep 19 11:09:20 2022] PUT /identity/v3/domains/default/users/dd316848081f4ae2ab8647785189b3f8/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:09:20.588738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e8f63-1597-494c-a1b6-c4881d1be4be None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:20.588738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e8f63-1597-494c-a1b6-c4881d1be4be None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:20.588738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e8f63-1597-494c-a1b6-c4881d1be4be None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:20.589105 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463c619-f416-459a-a18a-28d740f93aad None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:20.589105 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463c619-f416-459a-a18a-28d740f93aad None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:20.589253 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e8f63-1597-494c-a1b6-c4881d1be4be None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:20.589330 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463c619-f416-459a-a18a-28d740f93aad None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:20.589330 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 20/40] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:20.589330 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463c619-f416-459a-a18a-28d740f93aad None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:20.589545 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e8f63-1597-494c-a1b6-c4881d1be4be None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:20.723589 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463c619-f416-459a-a18a-28d740f93aad None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:20.738857 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 21/41] 10.222.0.81 () {60 vars in 1141 bytes} [Mon Sep 19 11:09:20 2022] POST /identity/v3/projects => generated 301 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:09:20.740188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5a6c0-c0b8-4864-ae4a-4e469dfc27aa None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:20.740478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5a6c0-c0b8-4864-ae4a-4e469dfc27aa None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:20.740754 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5a6c0-c0b8-4864-ae4a-4e469dfc27aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:20.761678 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-72c5a6c0-c0b8-4864-ae4a-4e469dfc27aa None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:20.763300 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-72c5a6c0-c0b8-4864-ae4a-4e469dfc27aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:20.776401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4d983-6282-499a-9031-c7caa9112120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:20.776960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4d983-6282-499a-9031-c7caa9112120 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:20.777933 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4d983-6282-499a-9031-c7caa9112120 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:20.777933 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4d983-6282-499a-9031-c7caa9112120 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:20.778205 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4d983-6282-499a-9031-c7caa9112120 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:20.779604 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4d983-6282-499a-9031-c7caa9112120 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:20.797518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 21/42] 10.222.0.81 () {60 vars in 1131 bytes} [Mon Sep 19 11:09:20 2022] POST /identity/v3/roles => generated 225 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:09:20.798424 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4d983-6282-499a-9031-c7caa9112120 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:20.844141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:20.849396 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 22/43] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:20.857311 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:21.040276 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4d983-6282-499a-9031-c7caa9112120 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:21.080675 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35739f-0bac-4f6c-b3bd-34d39d6968fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:21.081301 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35739f-0bac-4f6c-b3bd-34d39d6968fe None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:21.081587 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35739f-0bac-4f6c-b3bd-34d39d6968fe None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:21.081861 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35739f-0bac-4f6c-b3bd-34d39d6968fe None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:21.082537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c35739f-0bac-4f6c-b3bd-34d39d6968fe None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:21.083961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c35739f-0bac-4f6c-b3bd-34d39d6968fe None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:21.100010 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 22/44] 10.222.0.81 () {60 vars in 1131 bytes} [Mon Sep 19 11:09:20 2022] POST /identity/v3/roles => generated 223 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:09:21.191869 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5a6c0-c0b8-4864-ae4a-4e469dfc27aa None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:21.304173 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5a6c0-c0b8-4864-ae4a-4e469dfc27aa None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:21.339745 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace14a05-494b-4890-b3a2-a74b3d981ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:21.340405 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace14a05-494b-4890-b3a2-a74b3d981ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:21.340741 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace14a05-494b-4890-b3a2-a74b3d981ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:21.340956 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace14a05-494b-4890-b3a2-a74b3d981ac2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:21.342066 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace14a05-494b-4890-b3a2-a74b3d981ac2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:21.343648 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace14a05-494b-4890-b3a2-a74b3d981ac2 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:21.345226 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 23/45] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:21.350675 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35739f-0bac-4f6c-b3bd-34d39d6968fe None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:21.449717 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35739f-0bac-4f6c-b3bd-34d39d6968fe None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:21.451240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:21.485597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78ad32-49ed-478a-80ae-b3fd54c37d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:21.486209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78ad32-49ed-478a-80ae-b3fd54c37d8b None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:21.486493 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78ad32-49ed-478a-80ae-b3fd54c37d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:21.486766 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78ad32-49ed-478a-80ae-b3fd54c37d8b None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:21.487418 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78ad32-49ed-478a-80ae-b3fd54c37d8b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:21.488911 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78ad32-49ed-478a-80ae-b3fd54c37d8b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:21.516523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 23/46] 10.222.0.81 () {60 vars in 1141 bytes} [Mon Sep 19 11:09:21 2022] POST /identity/v3/projects => generated 312 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:09:22.902545 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9cdb2-e085-46eb-9b4e-7500554c4ab3 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:22.903095 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9cdb2-e085-46eb-9b4e-7500554c4ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:22.903095 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9cdb2-e085-46eb-9b4e-7500554c4ab3 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:22.904460 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 24/47] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:22.909799 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50b1f5-abd5-44e9-85f8-367d92a738ca None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:22.910199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50b1f5-abd5-44e9-85f8-367d92a738ca None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:22.910361 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50b1f5-abd5-44e9-85f8-367d92a738ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:22.929164 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8a50b1f5-abd5-44e9-85f8-367d92a738ca None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:22.930625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8a50b1f5-abd5-44e9-85f8-367d92a738ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:22.996008 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 24/48] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:23.269593 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9cdb2-e085-46eb-9b4e-7500554c4ab3 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:23.337217 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:23.371184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9cdb2-e085-46eb-9b4e-7500554c4ab3 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:23.405391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0d4d2-55f4-40d1-9820-4ce38e663ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:23.406024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0d4d2-55f4-40d1-9820-4ce38e663ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:23.406306 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0d4d2-55f4-40d1-9820-4ce38e663ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:23.406581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0d4d2-55f4-40d1-9820-4ce38e663ea7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:23.418339 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0d4d2-55f4-40d1-9820-4ce38e663ea7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:23.424659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-67a0d4d2-55f4-40d1-9820-4ce38e663ea7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:23.424659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:23.424659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:23.424659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:23.424659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:23.424659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:23.424659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:23.424659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:23.424659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:23.424659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:23.424659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:23.424659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:23.424659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:23.424659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:23.424659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:23.424659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:23.425485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:23.425485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:23.425485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:23.425485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:23.425485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:23.425485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:23.425485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:23.425485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:23.425485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:23.425485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:23.425485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:23.425485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:23.425485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:23.425485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:23.425485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:23.425485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:23.426243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:23.426243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:23.426243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:23.426243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:23.426243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:23.426243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:23.426243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:23.426243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:23.426243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:23.426243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:23.426243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:23.426243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:09:23.426243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 25/49] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:09:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:23.431575 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50b1f5-abd5-44e9-85f8-367d92a738ca None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:23.513801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:23.530222 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50b1f5-abd5-44e9-85f8-367d92a738ca None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:23.566227 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2c691-b4da-470b-be0a-cc3c9d98ac24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:23.566870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2c691-b4da-470b-be0a-cc3c9d98ac24 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:23.567150 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2c691-b4da-470b-be0a-cc3c9d98ac24 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:23.567427 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2c691-b4da-470b-be0a-cc3c9d98ac24 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:23.568161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee2c691-b4da-470b-be0a-cc3c9d98ac24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:23.568473 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee2c691-b4da-470b-be0a-cc3c9d98ac24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:23.569933 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee2c691-b4da-470b-be0a-cc3c9d98ac24 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:23.579245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 25/50] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:23.584853 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0d4d2-55f4-40d1-9820-4ce38e663ea7 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:23.663422 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e98a5-52df-4f99-91fd-3752387c654e None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:23.663708 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e98a5-52df-4f99-91fd-3752387c654e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:23.664019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e98a5-52df-4f99-91fd-3752387c654e None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:23.665359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 26/51] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:09:23.670890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43a4b1-70be-457b-9e8d-26d120baa84e None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:23.671712 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43a4b1-70be-457b-9e8d-26d120baa84e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:23.671712 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43a4b1-70be-457b-9e8d-26d120baa84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:23.689353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-1d43a4b1-70be-457b-9e8d-26d120baa84e None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:23.690557 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1d43a4b1-70be-457b-9e8d-26d120baa84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:23.692390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0d4d2-55f4-40d1-9820-4ce38e663ea7 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:23.719127 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:23.728220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0d84f-629b-4210-bfe5-057fc137f179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:23.728882 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0d84f-629b-4210-bfe5-057fc137f179 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:23.729170 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0d84f-629b-4210-bfe5-057fc137f179 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:23.729450 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0d84f-629b-4210-bfe5-057fc137f179 None admin] PATH_INFO: `/v3/users/dd316848081f4ae2ab8647785189b3f8` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:23.730615 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c0d84f-629b-4210-bfe5-057fc137f179 None admin] RBAC: Authorizing `identity:get_user(user_id=dd316848081f4ae2ab8647785189b3f8)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:23.732102 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c0d84f-629b-4210-bfe5-057fc137f179 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:23.733682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 26/52] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:23 2022] GET /identity/v3/users/dd316848081f4ae2ab8647785189b3f8 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:23.739189 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0d84f-629b-4210-bfe5-057fc137f179 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:23.761001 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 27/53] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:23.838933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0d84f-629b-4210-bfe5-057fc137f179 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:23.871043 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4084bfa-5e68-4056-a630-30760ee8aa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:23.871647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4084bfa-5e68-4056-a630-30760ee8aa87 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:23.871965 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4084bfa-5e68-4056-a630-30760ee8aa87 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:23.872248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4084bfa-5e68-4056-a630-30760ee8aa87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:23.873364 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4084bfa-5e68-4056-a630-30760ee8aa87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:23.874805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4084bfa-5e68-4056-a630-30760ee8aa87 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:23.876580 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 27/54] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:23.882327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43a4b1-70be-457b-9e8d-26d120baa84e None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:23.957152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:23.980493 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43a4b1-70be-457b-9e8d-26d120baa84e None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:24.014266 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d128b-de06-411b-a81e-acc7a93172a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:24.014867 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d128b-de06-411b-a81e-acc7a93172a6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:24.015152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d128b-de06-411b-a81e-acc7a93172a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:24.015431 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d128b-de06-411b-a81e-acc7a93172a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:24.016187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d128b-de06-411b-a81e-acc7a93172a6 None admin] RBAC: Adding query filter params (role.id=753f8b35d5f0458d8f62d0ad6e28033f, scope.domain.id=default, user.id=dd316848081f4ae2ab8647785189b3f8) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:24.016501 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d128b-de06-411b-a81e-acc7a93172a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:24.017967 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d128b-de06-411b-a81e-acc7a93172a6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:24.023786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 28/55] 10.222.0.81 () {56 vars in 1322 bytes} [Mon Sep 19 11:09:23 2022] GET /identity/v3/role_assignments?user.id=dd316848081f4ae2ab8647785189b3f8&scope.domain.id=default&role.id=753f8b35d5f0458d8f62d0ad6e28033f => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:24.026676 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4084bfa-5e68-4056-a630-30760ee8aa87 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:24.198587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:24.238389 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4084bfa-5e68-4056-a630-30760ee8aa87 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:24.272616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f61495d-c8e0-4e3e-817a-8470609aaf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:24.273284 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61495d-c8e0-4e3e-817a-8470609aaf55 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:24.273571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61495d-c8e0-4e3e-817a-8470609aaf55 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:24.273851 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61495d-c8e0-4e3e-817a-8470609aaf55 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:24.274968 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f61495d-c8e0-4e3e-817a-8470609aaf55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:24.277014 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f61495d-c8e0-4e3e-817a-8470609aaf55 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:24.278587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 28/56] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:24.284499 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d128b-de06-411b-a81e-acc7a93172a6 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:24.381042 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d128b-de06-411b-a81e-acc7a93172a6 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:24.413858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf5da5-38ad-4f58-a08a-167b838b14ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:24.414456 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf5da5-38ad-4f58-a08a-167b838b14ec None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:24.414740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf5da5-38ad-4f58-a08a-167b838b14ec None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:24.415016 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf5da5-38ad-4f58-a08a-167b838b14ec None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:24.415673 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf5da5-38ad-4f58-a08a-167b838b14ec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:24.417790 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf5da5-38ad-4f58-a08a-167b838b14ec None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:24.428521 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:24.443681 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 29/57] 10.222.0.81 () {60 vars in 1140 bytes} [Mon Sep 19 11:09:24 2022] POST /identity/v3/projects => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:09:26.612954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b124e-6d5e-4589-805b-ee06a9571b23 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:26.613484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b124e-6d5e-4589-805b-ee06a9571b23 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:26.613484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b124e-6d5e-4589-805b-ee06a9571b23 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:26.614844 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 29/58] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:26.621439 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350e20f-72d6-4ce2-aeaa-67042d1cfce2 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:26.621860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350e20f-72d6-4ce2-aeaa-67042d1cfce2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:26.621998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350e20f-72d6-4ce2-aeaa-67042d1cfce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:26.640390 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-b350e20f-72d6-4ce2-aeaa-67042d1cfce2 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:26.641602 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b350e20f-72d6-4ce2-aeaa-67042d1cfce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:26.706519 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 30/59] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:26.987112 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b124e-6d5e-4589-805b-ee06a9571b23 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:27.085759 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b124e-6d5e-4589-805b-ee06a9571b23 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:27.114604 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:27.120105 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c393f-564a-43eb-8490-710146c33b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:27.120727 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c393f-564a-43eb-8490-710146c33b50 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:27.121011 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c393f-564a-43eb-8490-710146c33b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:27.121288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c393f-564a-43eb-8490-710146c33b50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:27.122431 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c393f-564a-43eb-8490-710146c33b50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:27.123903 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c393f-564a-43eb-8490-710146c33b50 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:27.125448 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 30/60] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:27.130845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350e20f-72d6-4ce2-aeaa-67042d1cfce2 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:27.231450 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350e20f-72d6-4ce2-aeaa-67042d1cfce2 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:27.253339 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:27.266728 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199e03c-17d7-4ac7-8379-a013d1ee8bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:27.267347 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199e03c-17d7-4ac7-8379-a013d1ee8bee None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:27.267629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199e03c-17d7-4ac7-8379-a013d1ee8bee None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:27.267928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199e03c-17d7-4ac7-8379-a013d1ee8bee None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:27.268584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199e03c-17d7-4ac7-8379-a013d1ee8bee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:27.270049 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199e03c-17d7-4ac7-8379-a013d1ee8bee None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:27.297156 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 31/61] 10.222.0.81 () {60 vars in 1132 bytes} [Mon Sep 19 11:09:27 2022] POST /identity/v3/users => generated 275 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:09:30.656299 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0a509-5932-4c85-87cf-3e462a91a25e None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:30.656813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0a509-5932-4c85-87cf-3e462a91a25e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:30.656887 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0a509-5932-4c85-87cf-3e462a91a25e None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:30.658467 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 31/62] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:30.663974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d59f0-ce8d-4e03-a4ac-f4c540d874f0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:30.664439 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d59f0-ce8d-4e03-a4ac-f4c540d874f0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:30.664539 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d59f0-ce8d-4e03-a4ac-f4c540d874f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:30.693394 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a37d59f0-ce8d-4e03-a4ac-f4c540d874f0 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:30.694634 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a37d59f0-ce8d-4e03-a4ac-f4c540d874f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:30.712825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8946c-7794-4cb6-991c-21f33768ce9f None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:30.713166 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8946c-7794-4cb6-991c-21f33768ce9f None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:30.713916 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8946c-7794-4cb6-991c-21f33768ce9f None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:30.715256 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 32/63] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:30.720503 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ab6f1-0309-4b97-b2ef-ad302bff75f3 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:30.720794 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ab6f1-0309-4b97-b2ef-ad302bff75f3 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:30.721074 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ab6f1-0309-4b97-b2ef-ad302bff75f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:30.751933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-273ab6f1-0309-4b97-b2ef-ad302bff75f3 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:30.753188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-273ab6f1-0309-4b97-b2ef-ad302bff75f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:30.841823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 32/64] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:30.874654 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7be96-9772-45db-b500-5cc70f803d6d None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:30.875285 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7be96-9772-45db-b500-5cc70f803d6d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:30.875799 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7be96-9772-45db-b500-5cc70f803d6d None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:30.877362 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 33/65] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:09:30.882624 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f51d8-9670-4bbd-bcc8-912a5b441085 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:30.883150 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f51d8-9670-4bbd-bcc8-912a5b441085 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:30.883629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f51d8-9670-4bbd-bcc8-912a5b441085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:30.892474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 33/66] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:09:30.903498 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8c4f51d8-9670-4bbd-bcc8-912a5b441085 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:30.905000 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8c4f51d8-9670-4bbd-bcc8-912a5b441085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:30.996922 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 34/67] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:09:31.206497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40011b5-9454-4bf5-a95c-70fd91c8d74e None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:31.206856 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40011b5-9454-4bf5-a95c-70fd91c8d74e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:31.207065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40011b5-9454-4bf5-a95c-70fd91c8d74e None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:31.208104 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d571ad1-8a0f-49ce-8682-8c8b9cb3b16f None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:31.208600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d571ad1-8a0f-49ce-8682-8c8b9cb3b16f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:31.208770 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 34/68] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:31.209233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d571ad1-8a0f-49ce-8682-8c8b9cb3b16f None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:31.210861 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 35/69] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:31.213870 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcea773-dc3a-493c-a199-6207aec221a0 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:31.214160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcea773-dc3a-493c-a199-6207aec221a0 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:31.214437 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcea773-dc3a-493c-a199-6207aec221a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:31.217296 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24e8ea-d321-4f32-bc00-d8d58564fbc4 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:31.217983 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24e8ea-d321-4f32-bc00-d8d58564fbc4 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:31.218460 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24e8ea-d321-4f32-bc00-d8d58564fbc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:31.236900 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-bb24e8ea-d321-4f32-bc00-d8d58564fbc4 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:31.238407 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bb24e8ea-d321-4f32-bc00-d8d58564fbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:31.244803 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-afcea773-dc3a-493c-a199-6207aec221a0 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:31.246513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-afcea773-dc3a-493c-a199-6207aec221a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:31.294257 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:31.318288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 36/70] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:09:31.319169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24e8ea-d321-4f32-bc00-d8d58564fbc4 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:31.343357 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 35/71] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:31.348916 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcea773-dc3a-493c-a199-6207aec221a0 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:31.431920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:31.451285 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24e8ea-d321-4f32-bc00-d8d58564fbc4 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:31.489168 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f523e-6c98-4ca9-863b-40bb10052b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:31.490062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f523e-6c98-4ca9-863b-40bb10052b6d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:31.490553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f523e-6c98-4ca9-863b-40bb10052b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:31.491048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f523e-6c98-4ca9-863b-40bb10052b6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:31.492605 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcea773-dc3a-493c-a199-6207aec221a0 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:31.502376 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f523e-6c98-4ca9-863b-40bb10052b6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:31.507660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-274f523e-6c98-4ca9-863b-40bb10052b6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:09:31.507660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:31.507660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:31.507660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:31.507660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:31.507660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:31.507660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:31.507660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:31.507660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:31.507660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:31.507660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:31.507660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:31.507660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:31.507660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:31.507660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:31.507660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:31.508651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:31.508651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:31.508651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:31.508651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:31.508651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:31.508651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:31.508651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:31.508651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:31.508651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:31.508651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:31.508651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:31.508651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:31.508651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:31.508651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:31.508651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:31.508651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:31.509421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:31.509421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:31.509421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:31.509421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:31.509421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:31.509421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:31.509421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:31.509421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:31.509421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:31.509421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:31.509421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:09:31.509421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:09:31.512234 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 37/72] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:31.513446 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f523e-6c98-4ca9-863b-40bb10052b6d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:31.536190 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cb115-b8ce-460a-8356-dca094c86443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:31.536869 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cb115-b8ce-460a-8356-dca094c86443 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:31.537194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cb115-b8ce-460a-8356-dca094c86443 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:31.537500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cb115-b8ce-460a-8356-dca094c86443 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:31.547041 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:31.552715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05cb115-b8ce-460a-8356-dca094c86443 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:31.559070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-f05cb115-b8ce-460a-8356-dca094c86443 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:31.559070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:31.559070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:31.559070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:31.559070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:31.559070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:31.559070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:31.559070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:31.559070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:31.559070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:31.559070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:31.559070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:31.559070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:31.559070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:31.559070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:31.559070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:31.559952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:31.559952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:31.559952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:31.559952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:31.559952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:31.559952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:31.559952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:31.559952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:31.559952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:31.559952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:31.559952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:31.559952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:31.559952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:31.559952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:31.559952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:31.559952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:31.561136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:31.561136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:31.561136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:31.561136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:31.561136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:31.561136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:31.561136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:31.561136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:31.561136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:31.561136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:31.561136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:31.561136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:09:31.561136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 36/73] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:09:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:31.562109 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cb115-b8ce-460a-8356-dca094c86443 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:31.633056 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f523e-6c98-4ca9-863b-40bb10052b6d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:31.668078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88183035-ea2e-467a-a694-37875e5a151a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:31.669048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88183035-ea2e-467a-a694-37875e5a151a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:31.669048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88183035-ea2e-467a-a694-37875e5a151a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:31.669288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88183035-ea2e-467a-a694-37875e5a151a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:31.679866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88183035-ea2e-467a-a694-37875e5a151a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:31.686002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-88183035-ea2e-467a-a694-37875e5a151a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:09:31.686002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:31.686002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:31.686002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:31.686002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:31.686002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:31.686002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:31.686002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:31.686002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:31.686002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:31.686002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:31.686002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:31.686002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:31.686002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:31.686002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:31.686002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:31.686932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:31.686932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:31.686932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:31.686932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:31.686932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:31.686932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:31.686932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:31.686932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:31.686932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:31.686932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:31.686932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:31.686932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:31.686932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:31.686932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:31.686932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:31.686932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:31.694165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:31.694165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:31.694165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:31.694165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:31.694165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:31.694165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:31.694165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:31.694165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:31.694165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:31.694165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:31.694165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:09:31.694165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:09:31.694779 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cb115-b8ce-460a-8356-dca094c86443 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:31.695192 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 38/74] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:09:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:31.695192 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88183035-ea2e-467a-a694-37875e5a151a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:31.709516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:31.724659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d52cef-8d39-4a94-9e9b-033609297b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:31.725265 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d52cef-8d39-4a94-9e9b-033609297b13 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:31.725546 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d52cef-8d39-4a94-9e9b-033609297b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:31.725819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d52cef-8d39-4a94-9e9b-033609297b13 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:31.726480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d52cef-8d39-4a94-9e9b-033609297b13 None admin] RBAC: Adding query filter params (name=member) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:31.726784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d52cef-8d39-4a94-9e9b-033609297b13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:31.728192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d52cef-8d39-4a94-9e9b-033609297b13 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:31.736704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 37/75] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:09:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:31.737487 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d52cef-8d39-4a94-9e9b-033609297b13 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:31.785893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:31.799842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88183035-ea2e-467a-a694-37875e5a151a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:31.833977 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d254e1-fa06-4657-979f-00f1c9afd46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:31.834584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d254e1-fa06-4657-979f-00f1c9afd46f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:31.834865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d254e1-fa06-4657-979f-00f1c9afd46f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:31.835142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d254e1-fa06-4657-979f-00f1c9afd46f None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:31.835841 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d254e1-fa06-4657-979f-00f1c9afd46f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:31.836149 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d254e1-fa06-4657-979f-00f1c9afd46f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:31.837107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d52cef-8d39-4a94-9e9b-033609297b13 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:31.837554 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d254e1-fa06-4657-979f-00f1c9afd46f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:31.845808 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 39/76] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:31.846932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d254e1-fa06-4657-979f-00f1c9afd46f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:31.874712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe160f4e-d78d-4565-8db2-393f2a6d6463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:31.875344 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe160f4e-d78d-4565-8db2-393f2a6d6463 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:31.875622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe160f4e-d78d-4565-8db2-393f2a6d6463 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:31.875929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe160f4e-d78d-4565-8db2-393f2a6d6463 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:31.877088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe160f4e-d78d-4565-8db2-393f2a6d6463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:31.878764 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe160f4e-d78d-4565-8db2-393f2a6d6463 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:31.880411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 38/77] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:31.881199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe160f4e-d78d-4565-8db2-393f2a6d6463 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:31.931602 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:31.948691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d254e1-fa06-4657-979f-00f1c9afd46f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:31.982018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c8945-9df2-4329-91e8-12313afa87bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:31.982645 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c8945-9df2-4329-91e8-12313afa87bb None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:31.982927 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c8945-9df2-4329-91e8-12313afa87bb None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:31.983202 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c8945-9df2-4329-91e8-12313afa87bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:31.984368 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe160f4e-d78d-4565-8db2-393f2a6d6463 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:31.993192 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c8945-9df2-4329-91e8-12313afa87bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:31.998989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-db3c8945-9df2-4329-91e8-12313afa87bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:09:31.998989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:31.998989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:31.998989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:31.998989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:31.998989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:31.998989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:31.998989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:31.998989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:31.998989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:31.998989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:31.998989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:31.998989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:31.998989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:31.998989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:31.998989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:32.000101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:32.000101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:32.000101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:32.000101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:32.000101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:32.000101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:32.000101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:32.000101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:32.000101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:32.000101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:32.000101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:32.000101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:32.000101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:32.000101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:32.000101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:32.000101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:32.001004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:32.001004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:32.001004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:32.001004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:32.001004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:32.001004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:32.001004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:32.001004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:32.001004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:32.001004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:32.001004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:09:32.001004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:09:32.001004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 40/78] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:32.001004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c8945-9df2-4329-91e8-12313afa87bb None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:32.013134 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:32.022339 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4bd744-52eb-420a-bc29-ba1e9f37d7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:32.022951 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bd744-52eb-420a-bc29-ba1e9f37d7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:32.023239 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bd744-52eb-420a-bc29-ba1e9f37d7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:32.023520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bd744-52eb-420a-bc29-ba1e9f37d7e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:32.024218 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4bd744-52eb-420a-bc29-ba1e9f37d7e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:32.024524 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4bd744-52eb-420a-bc29-ba1e9f37d7e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:32.025933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4bd744-52eb-420a-bc29-ba1e9f37d7e7 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:32.034028 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 39/79] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:09:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:32.038746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4bd744-52eb-420a-bc29-ba1e9f37d7e7 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:32.103180 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c8945-9df2-4329-91e8-12313afa87bb None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:32.135215 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4bd744-52eb-420a-bc29-ba1e9f37d7e7 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:32.138150 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99011c37-bfc0-4558-848e-ea3865dd47e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:32.138782 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99011c37-bfc0-4558-848e-ea3865dd47e9 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:32.139068 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99011c37-bfc0-4558-848e-ea3865dd47e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:32.139349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99011c37-bfc0-4558-848e-ea3865dd47e9 None admin] PATH_INFO: `/v3/users/6aeda366bbb94f7cb91aa6788d9285e7` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:32.147576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:32.152607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99011c37-bfc0-4558-848e-ea3865dd47e9 None admin] RBAC: Authorizing `identity:get_user(user_id=6aeda366bbb94f7cb91aa6788d9285e7)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:32.154079 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99011c37-bfc0-4558-848e-ea3865dd47e9 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:32.155896 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 41/80] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:32 2022] GET /identity/v3/users/6aeda366bbb94f7cb91aa6788d9285e7 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:32.156509 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99011c37-bfc0-4558-848e-ea3865dd47e9 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:32.169621 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06191015-fff3-417c-b0c4-72eef972b8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:32.170249 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06191015-fff3-417c-b0c4-72eef972b8cf None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:32.170528 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06191015-fff3-417c-b0c4-72eef972b8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:32.170801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06191015-fff3-417c-b0c4-72eef972b8cf None admin] PATH_INFO: `/v3/users/dd316848081f4ae2ab8647785189b3f8` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:32.171958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06191015-fff3-417c-b0c4-72eef972b8cf None admin] RBAC: Authorizing `identity:get_user(user_id=dd316848081f4ae2ab8647785189b3f8)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:32.173375 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06191015-fff3-417c-b0c4-72eef972b8cf None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:32.174931 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 40/81] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:32 2022] GET /identity/v3/users/dd316848081f4ae2ab8647785189b3f8 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:32.175681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06191015-fff3-417c-b0c4-72eef972b8cf None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:32.228551 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:32.263165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99011c37-bfc0-4558-848e-ea3865dd47e9 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:32.276509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06191015-fff3-417c-b0c4-72eef972b8cf None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:32.297116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f702d-56d2-41e5-9c23-745f21ae93cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:32.297719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f702d-56d2-41e5-9c23-745f21ae93cc None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:32.298002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f702d-56d2-41e5-9c23-745f21ae93cc None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:32.298275 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f702d-56d2-41e5-9c23-745f21ae93cc None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:32.298929 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f702d-56d2-41e5-9c23-745f21ae93cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:32.300402 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f702d-56d2-41e5-9c23-745f21ae93cc None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:32.310417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd649189-ea24-4115-a858-aa0b4b8fbf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:32.311020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd649189-ea24-4115-a858-aa0b4b8fbf8a None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:32.311297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd649189-ea24-4115-a858-aa0b4b8fbf8a None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:32.311569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd649189-ea24-4115-a858-aa0b4b8fbf8a None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:32.312263 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd649189-ea24-4115-a858-aa0b4b8fbf8a None admin] RBAC: Adding query filter params (name=member) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:32.312562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd649189-ea24-4115-a858-aa0b4b8fbf8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:32.314187 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd649189-ea24-4115-a858-aa0b4b8fbf8a None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:32.323596 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 41/82] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:09:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:32.323885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 42/83] 10.222.0.81 () {60 vars in 1132 bytes} [Mon Sep 19 11:09:32 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:09:32.324588 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f702d-56d2-41e5-9c23-745f21ae93cc None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:32.329980 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd649189-ea24-4115-a858-aa0b4b8fbf8a None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:32.380345 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:32.448377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd649189-ea24-4115-a858-aa0b4b8fbf8a None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:32.453244 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f702d-56d2-41e5-9c23-745f21ae93cc None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:32.481210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:32.489436 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2404be-3533-4822-bd3f-0dbb80446479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:32.490134 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2404be-3533-4822-bd3f-0dbb80446479 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:32.490422 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2404be-3533-4822-bd3f-0dbb80446479 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:32.490704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2404be-3533-4822-bd3f-0dbb80446479 None admin] PATH_INFO: `/v3/users/6aeda366bbb94f7cb91aa6788d9285e7` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:32.491924 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2404be-3533-4822-bd3f-0dbb80446479 None admin] RBAC: Authorizing `identity:get_user(user_id=6aeda366bbb94f7cb91aa6788d9285e7)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:32.493391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2404be-3533-4822-bd3f-0dbb80446479 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:32.495562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 42/84] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:32 2022] GET /identity/v3/users/6aeda366bbb94f7cb91aa6788d9285e7 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:32.496461 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2404be-3533-4822-bd3f-0dbb80446479 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:32.498372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5ec0a-3248-4677-9c1e-ec1b4b4fee72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:32.499334 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5ec0a-3248-4677-9c1e-ec1b4b4fee72 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:32.499934 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5ec0a-3248-4677-9c1e-ec1b4b4fee72 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:32.500215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5ec0a-3248-4677-9c1e-ec1b4b4fee72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:32.581480 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd5ec0a-3248-4677-9c1e-ec1b4b4fee72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:32.582970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd5ec0a-3248-4677-9c1e-ec1b4b4fee72 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:32.615340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2404be-3533-4822-bd3f-0dbb80446479 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:32.633011 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:32.663075 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de967d-004c-4a1a-a1b1-e4580f8aea04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:32.664065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de967d-004c-4a1a-a1b1-e4580f8aea04 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:32.664499 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de967d-004c-4a1a-a1b1-e4580f8aea04 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:32.664776 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de967d-004c-4a1a-a1b1-e4580f8aea04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:32.713451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 43/85] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:09:32.714476 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5ec0a-3248-4677-9c1e-ec1b4b4fee72 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:32.746636 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:32.765814 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de967d-004c-4a1a-a1b1-e4580f8aea04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:32.767669 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de967d-004c-4a1a-a1b1-e4580f8aea04 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:32.833068 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5ec0a-3248-4677-9c1e-ec1b4b4fee72 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:32.869163 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f05ccf-6cae-4072-8047-4be1b5158021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:32.870160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f05ccf-6cae-4072-8047-4be1b5158021 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:32.870638 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f05ccf-6cae-4072-8047-4be1b5158021 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:32.871124 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f05ccf-6cae-4072-8047-4be1b5158021 None admin] PATH_INFO: `/v3/users/6aeda366bbb94f7cb91aa6788d9285e7` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:32.872497 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f05ccf-6cae-4072-8047-4be1b5158021 None admin] RBAC: Authorizing `identity:get_user(user_id=6aeda366bbb94f7cb91aa6788d9285e7)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:32.874169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f05ccf-6cae-4072-8047-4be1b5158021 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:32.876085 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 44/86] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:32 2022] GET /identity/v3/users/6aeda366bbb94f7cb91aa6788d9285e7 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:32.880628 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 43/87] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:09:32.881265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f05ccf-6cae-4072-8047-4be1b5158021 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:32.881370 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de967d-004c-4a1a-a1b1-e4580f8aea04 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:32.894544 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:32.985925 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:32.997173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f05ccf-6cae-4072-8047-4be1b5158021 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:33.008141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de967d-004c-4a1a-a1b1-e4580f8aea04 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:33.037320 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fd1f3-072f-4dba-bffe-1e8f27d9aaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:33.038270 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fd1f3-072f-4dba-bffe-1e8f27d9aaaf None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:33.038761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fd1f3-072f-4dba-bffe-1e8f27d9aaaf None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:33.039224 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fd1f3-072f-4dba-bffe-1e8f27d9aaaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:33.053837 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f568673-51b5-492d-a9ce-fdc836480bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:33.054472 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f568673-51b5-492d-a9ce-fdc836480bc9 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:33.054755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f568673-51b5-492d-a9ce-fdc836480bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:33.055033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f568673-51b5-492d-a9ce-fdc836480bc9 None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:33.056465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f568673-51b5-492d-a9ce-fdc836480bc9 None admin] RBAC: Authorizing `identity:get_project(project_id=52fe574d05694a42951535f0d8667f77)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:33.059197 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f568673-51b5-492d-a9ce-fdc836480bc9 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:33.063896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 44/88] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:32 2022] GET /identity/v3/projects/52fe574d05694a42951535f0d8667f77 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:33.064884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f568673-51b5-492d-a9ce-fdc836480bc9 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:33.140531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895fd1f3-072f-4dba-bffe-1e8f27d9aaaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:33.142019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895fd1f3-072f-4dba-bffe-1e8f27d9aaaf None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:33.159880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:33.189581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f568673-51b5-492d-a9ce-fdc836480bc9 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:33.236973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9504c5-9f6b-4007-923e-e9aa783619ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:33.237610 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9504c5-9f6b-4007-923e-e9aa783619ad None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:33.237894 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9504c5-9f6b-4007-923e-e9aa783619ad None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:33.238408 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9504c5-9f6b-4007-923e-e9aa783619ad None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:33.239987 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9504c5-9f6b-4007-923e-e9aa783619ad None admin] RBAC: Authorizing `identity:get_project(project_id=52fe574d05694a42951535f0d8667f77)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:33.241434 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9504c5-9f6b-4007-923e-e9aa783619ad None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:33.243654 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 45/89] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:33 2022] GET /identity/v3/projects/52fe574d05694a42951535f0d8667f77 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:33.244496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9504c5-9f6b-4007-923e-e9aa783619ad None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:33.247530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:33.260760 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 45/90] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:09:33.261975 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fd1f3-072f-4dba-bffe-1e8f27d9aaaf None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:33.374471 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fd1f3-072f-4dba-bffe-1e8f27d9aaaf None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:33.391722 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9504c5-9f6b-4007-923e-e9aa783619ad None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:33.411623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0b443-6314-461c-808b-59993e685dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:33.412551 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0b443-6314-461c-808b-59993e685dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:33.413070 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0b443-6314-461c-808b-59993e685dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:33.413532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0b443-6314-461c-808b-59993e685dc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:33.414459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0b443-6314-461c-808b-59993e685dc4 None admin] RBAC: Adding query filter params (role.id=0c30c75e227c416da6e4814bce700e8d, scope.project.id=52fe574d05694a42951535f0d8667f77, user.id=6aeda366bbb94f7cb91aa6788d9285e7) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:33.414969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0b443-6314-461c-808b-59993e685dc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:33.417309 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0b443-6314-461c-808b-59993e685dc4 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:33.423130 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 46/91] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:09:33 2022] GET /identity/v3/role_assignments?user.id=6aeda366bbb94f7cb91aa6788d9285e7&scope.project.id=52fe574d05694a42951535f0d8667f77&role.id=0c30c75e227c416da6e4814bce700e8d => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:33.424191 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0b443-6314-461c-808b-59993e685dc4 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:33.450893 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3a3e0-3408-4e47-807e-9d5f58fb59b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:33.452185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3a3e0-3408-4e47-807e-9d5f58fb59b9 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:33.454820 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3a3e0-3408-4e47-807e-9d5f58fb59b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:33.455401 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3a3e0-3408-4e47-807e-9d5f58fb59b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:33.484687 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:33.542713 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:33.600509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3a3e0-3408-4e47-807e-9d5f58fb59b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:33.602874 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3a3e0-3408-4e47-807e-9d5f58fb59b9 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:33.629223 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0b443-6314-461c-808b-59993e685dc4 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:33.712764 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95f5c2-cf0f-4611-9d81-5b76423a3948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:33.713979 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95f5c2-cf0f-4611-9d81-5b76423a3948 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:33.714518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95f5c2-cf0f-4611-9d81-5b76423a3948 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:33.714518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95f5c2-cf0f-4611-9d81-5b76423a3948 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:33.715261 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95f5c2-cf0f-4611-9d81-5b76423a3948 None admin] RBAC: Adding query filter params (role.id=753f8b35d5f0458d8f62d0ad6e28033f, scope.project.id=52fe574d05694a42951535f0d8667f77, user.id=dd316848081f4ae2ab8647785189b3f8) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:33.715574 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95f5c2-cf0f-4611-9d81-5b76423a3948 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:33.719446 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95f5c2-cf0f-4611-9d81-5b76423a3948 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:33.725430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 47/92] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:09:33 2022] GET /identity/v3/role_assignments?user.id=dd316848081f4ae2ab8647785189b3f8&scope.project.id=52fe574d05694a42951535f0d8667f77&role.id=753f8b35d5f0458d8f62d0ad6e28033f => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:33.726260 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95f5c2-cf0f-4611-9d81-5b76423a3948 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:33.740578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 46/93] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:09:33.747001 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3a3e0-3408-4e47-807e-9d5f58fb59b9 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:33.880994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:33.916649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95f5c2-cf0f-4611-9d81-5b76423a3948 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:33.948557 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:33.959610 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402e489-d31c-47a8-a9ba-a7f624ed0855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:33.960237 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402e489-d31c-47a8-a9ba-a7f624ed0855 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:33.960526 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402e489-d31c-47a8-a9ba-a7f624ed0855 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:33.960808 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402e489-d31c-47a8-a9ba-a7f624ed0855 None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:33.966690 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402e489-d31c-47a8-a9ba-a7f624ed0855 None admin] RBAC: Authorizing `identity:get_project(project_id=52fe574d05694a42951535f0d8667f77)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:33.968229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402e489-d31c-47a8-a9ba-a7f624ed0855 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:33.970523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 48/94] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:33 2022] GET /identity/v3/projects/52fe574d05694a42951535f0d8667f77 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:33.975754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402e489-d31c-47a8-a9ba-a7f624ed0855 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:34.016170 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3a3e0-3408-4e47-807e-9d5f58fb59b9 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:34.083029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5625b-369b-478c-bc0e-310bf6c59678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:34.083669 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5625b-369b-478c-bc0e-310bf6c59678 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:34.083977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5625b-369b-478c-bc0e-310bf6c59678 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:34.084256 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5625b-369b-478c-bc0e-310bf6c59678 None admin] PATH_INFO: `/v3/projects/81a6489a782a48058c995b5765cdc625` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:34.086032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5625b-369b-478c-bc0e-310bf6c59678 None admin] RBAC: Authorizing `identity:get_project(project_id=81a6489a782a48058c995b5765cdc625)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:34.087465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5625b-369b-478c-bc0e-310bf6c59678 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:34.097870 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 47/95] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:33 2022] GET /identity/v3/projects/81a6489a782a48058c995b5765cdc625 => generated 312 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:34.104577 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5625b-369b-478c-bc0e-310bf6c59678 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:34.177230 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402e489-d31c-47a8-a9ba-a7f624ed0855 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:34.236558 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3ff23-b188-4225-beeb-03c616a046f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:34.237095 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3ff23-b188-4225-beeb-03c616a046f1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:34.237379 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3ff23-b188-4225-beeb-03c616a046f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:34.237661 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3ff23-b188-4225-beeb-03c616a046f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:34.238408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3ff23-b188-4225-beeb-03c616a046f1 None admin] RBAC: Adding query filter params (role.id=4cc230d31ac94f4c89bf827491c01bd7, scope.project.id=52fe574d05694a42951535f0d8667f77, user.id=6aeda366bbb94f7cb91aa6788d9285e7) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:34.238723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3ff23-b188-4225-beeb-03c616a046f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:34.240231 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3ff23-b188-4225-beeb-03c616a046f1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:34.256269 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 49/96] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:09:33 2022] GET /identity/v3/role_assignments?user.id=6aeda366bbb94f7cb91aa6788d9285e7&scope.project.id=52fe574d05694a42951535f0d8667f77&role.id=4cc230d31ac94f4c89bf827491c01bd7 => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:34.301884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5625b-369b-478c-bc0e-310bf6c59678 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:34.323255 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:34.355119 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c522918-29f6-4d10-b758-60a6dbb5f2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:34.355756 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c522918-29f6-4d10-b758-60a6dbb5f2bf None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:34.356055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c522918-29f6-4d10-b758-60a6dbb5f2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:34.356325 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c522918-29f6-4d10-b758-60a6dbb5f2bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:34.357087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c522918-29f6-4d10-b758-60a6dbb5f2bf None admin] RBAC: Adding query filter params (role.id=0c30c75e227c416da6e4814bce700e8d, scope.project.id=81a6489a782a48058c995b5765cdc625, user.id=6aeda366bbb94f7cb91aa6788d9285e7) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:34.357396 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c522918-29f6-4d10-b758-60a6dbb5f2bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:34.358879 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c522918-29f6-4d10-b758-60a6dbb5f2bf None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:34.371368 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 48/97] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:09:34 2022] GET /identity/v3/role_assignments?user.id=6aeda366bbb94f7cb91aa6788d9285e7&scope.project.id=81a6489a782a48058c995b5765cdc625&role.id=0c30c75e227c416da6e4814bce700e8d => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:34.934662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8ca17-150e-43bf-9037-3c049b90ef0b None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:34.935159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8ca17-150e-43bf-9037-3c049b90ef0b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:34.935252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab8ca17-150e-43bf-9037-3c049b90ef0b None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:34.936630 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 50/98] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:34.949169 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65abf1-4784-41d6-83b9-cdf479832ca3 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:34.949656 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65abf1-4784-41d6-83b9-cdf479832ca3 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:34.949729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65abf1-4784-41d6-83b9-cdf479832ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:34.973205 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-cb65abf1-4784-41d6-83b9-cdf479832ca3 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:34.974395 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-cb65abf1-4784-41d6-83b9-cdf479832ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:35.057199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 49/99] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:35.375321 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd166a0-bd6e-46a1-922e-3a3e57b3b102 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:35.375688 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd166a0-bd6e-46a1-922e-3a3e57b3b102 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:35.375916 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd166a0-bd6e-46a1-922e-3a3e57b3b102 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:35.377531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 51/100] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:35.382547 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad135d21-9e3d-4b07-a540-192dbe58c917 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:35.382834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad135d21-9e3d-4b07-a540-192dbe58c917 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:35.383109 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad135d21-9e3d-4b07-a540-192dbe58c917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:35.411791 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-ad135d21-9e3d-4b07-a540-192dbe58c917 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:35.412990 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ad135d21-9e3d-4b07-a540-192dbe58c917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:35.503635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 50/101] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:35.569274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd166a0-bd6e-46a1-922e-3a3e57b3b102 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:35.649405 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:35.726559 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd166a0-bd6e-46a1-922e-3a3e57b3b102 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:35.773790 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc478dd-85f0-4ba2-a745-a3a6999e3c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:35.774421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc478dd-85f0-4ba2-a745-a3a6999e3c62 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:35.774703 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc478dd-85f0-4ba2-a745-a3a6999e3c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:35.774979 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc478dd-85f0-4ba2-a745-a3a6999e3c62 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:35.785156 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc478dd-85f0-4ba2-a745-a3a6999e3c62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:35.790412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-5fc478dd-85f0-4ba2-a745-a3a6999e3c62 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:09:35.790412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:35.790412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:35.790412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:35.790412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:35.790412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:35.790412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:35.790412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:35.790412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:35.790412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:35.790412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:35.790412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:35.790412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:35.790412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:35.790412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:35.790412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:35.791852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:35.791852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:35.791852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:35.791852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:35.791852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:35.791852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:35.791852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:35.791852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:35.791852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:35.791852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:35.791852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:35.791852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:35.791852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:35.791852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:35.791852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:35.791852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:35.792691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:35.792691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:35.792691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:35.792691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:35.792691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:35.792691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:35.792691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:35.792691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:35.792691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:35.792691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:35.792691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:09:35.792691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:09:35.792691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 52/102] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:35.797757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad135d21-9e3d-4b07-a540-192dbe58c917 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:35.837294 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:35.915943 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad135d21-9e3d-4b07-a540-192dbe58c917 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:35.961639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cc2b2-3b72-4df0-b0de-03b5e872bc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:35.962274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cc2b2-3b72-4df0-b0de-03b5e872bc0e None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:35.962510 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cc2b2-3b72-4df0-b0de-03b5e872bc0e None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:35.962783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cc2b2-3b72-4df0-b0de-03b5e872bc0e None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:35.963538 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cc2b2-3b72-4df0-b0de-03b5e872bc0e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:35.963868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cc2b2-3b72-4df0-b0de-03b5e872bc0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:35.965258 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cc2b2-3b72-4df0-b0de-03b5e872bc0e None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:35.973312 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 51/103] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:09:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:35.979432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc478dd-85f0-4ba2-a745-a3a6999e3c62 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:36.064434 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cc2b2-3b72-4df0-b0de-03b5e872bc0e None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:36.112729 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc478dd-85f0-4ba2-a745-a3a6999e3c62 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:36.138582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:36.195861 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b732aef-55cb-42f0-85c9-3c21c885a341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:36.196373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b732aef-55cb-42f0-85c9-3c21c885a341 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:36.196660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b732aef-55cb-42f0-85c9-3c21c885a341 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:36.196941 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b732aef-55cb-42f0-85c9-3c21c885a341 None admin] PATH_INFO: `/v3/users/6aae10ffa9a84f9d901cc774261a3a09` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:36.211578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:36.216308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cc2b2-3b72-4df0-b0de-03b5e872bc0e None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:36.226083 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b732aef-55cb-42f0-85c9-3c21c885a341 None admin] RBAC: Authorizing `identity:get_user(user_id=6aae10ffa9a84f9d901cc774261a3a09)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:36.227624 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b732aef-55cb-42f0-85c9-3c21c885a341 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:36.229508 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 53/104] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:35 2022] GET /identity/v3/users/6aae10ffa9a84f9d901cc774261a3a09 => generated 289 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:36.235567 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b732aef-55cb-42f0-85c9-3c21c885a341 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:36.267973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4b0b6-ad0a-462a-af2a-c8bf7178cc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:36.269078 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4b0b6-ad0a-462a-af2a-c8bf7178cc25 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:36.269567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4b0b6-ad0a-462a-af2a-c8bf7178cc25 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:36.270043 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4b0b6-ad0a-462a-af2a-c8bf7178cc25 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:36.276571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4b0b6-ad0a-462a-af2a-c8bf7178cc25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:36.278382 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4b0b6-ad0a-462a-af2a-c8bf7178cc25 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:36.280389 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 52/105] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:36.285893 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4b0b6-ad0a-462a-af2a-c8bf7178cc25 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:36.438119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b732aef-55cb-42f0-85c9-3c21c885a341 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:36.506552 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4b0b6-ad0a-462a-af2a-c8bf7178cc25 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:36.514560 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961cdf9-39cc-4f6c-81b9-fe8a7a3e847d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:36.515182 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961cdf9-39cc-4f6c-81b9-fe8a7a3e847d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:36.515468 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961cdf9-39cc-4f6c-81b9-fe8a7a3e847d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:36.515749 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961cdf9-39cc-4f6c-81b9-fe8a7a3e847d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:36.568673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96a4ee-0feb-4519-9f46-275abb83facd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:36.569710 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96a4ee-0feb-4519-9f46-275abb83facd None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:36.570224 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96a4ee-0feb-4519-9f46-275abb83facd None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:36.570715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96a4ee-0feb-4519-9f46-275abb83facd None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:36.571582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc96a4ee-0feb-4519-9f46-275abb83facd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:36.573797 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc96a4ee-0feb-4519-9f46-275abb83facd None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:36.597752 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:36.643789 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 53/106] 10.222.0.81 () {60 vars in 1141 bytes} [Mon Sep 19 11:09:36 2022] POST /identity/v3/projects => generated 302 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:09:36.650207 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2961cdf9-39cc-4f6c-81b9-fe8a7a3e847d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:36.653752 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2961cdf9-39cc-4f6c-81b9-fe8a7a3e847d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:36.754784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021eb51-b45d-4b24-8ae6-9d57275f718e None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:36.755110 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021eb51-b45d-4b24-8ae6-9d57275f718e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:36.755347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021eb51-b45d-4b24-8ae6-9d57275f718e None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:36.757119 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 54/107] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:36.763261 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1294594-2272-4989-a81b-c8f587502f7a None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:36.763540 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1294594-2272-4989-a81b-c8f587502f7a None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:36.763848 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1294594-2272-4989-a81b-c8f587502f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:36.785525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-f1294594-2272-4989-a81b-c8f587502f7a None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:36.786746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f1294594-2272-4989-a81b-c8f587502f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:36.817366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:36.848813 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 54/108] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:09:36.854053 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961cdf9-39cc-4f6c-81b9-fe8a7a3e847d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:36.924173 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 55/109] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:09:36.972117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:36.999074 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961cdf9-39cc-4f6c-81b9-fe8a7a3e847d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:37.043034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a000f20-bfed-4229-b634-efdbd0b533d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:37.044030 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a000f20-bfed-4229-b634-efdbd0b533d3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:37.044536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a000f20-bfed-4229-b634-efdbd0b533d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:37.045028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a000f20-bfed-4229-b634-efdbd0b533d3 None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:37.052479 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a000f20-bfed-4229-b634-efdbd0b533d3 None admin] RBAC: Authorizing `identity:get_project(project_id=52fe574d05694a42951535f0d8667f77)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:37.054190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a000f20-bfed-4229-b634-efdbd0b533d3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:37.056241 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 55/110] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:36 2022] GET /identity/v3/projects/52fe574d05694a42951535f0d8667f77 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:37.063597 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1294594-2272-4989-a81b-c8f587502f7a None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:37.095340 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eee712-8b8a-4b37-ab3a-72b5e9e4c8fc None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:37.095651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eee712-8b8a-4b37-ab3a-72b5e9e4c8fc None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:37.095948 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eee712-8b8a-4b37-ab3a-72b5e9e4c8fc None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:37.097301 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 56/111] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:37.102564 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c163b-ae82-4dc6-8347-7c53d9c1d582 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:37.102851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c163b-ae82-4dc6-8347-7c53d9c1d582 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:37.103128 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c163b-ae82-4dc6-8347-7c53d9c1d582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:37.137865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-3b2c163b-ae82-4dc6-8347-7c53d9c1d582 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:37.139088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3b2c163b-ae82-4dc6-8347-7c53d9c1d582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:37.245885 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1294594-2272-4989-a81b-c8f587502f7a None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:37.268279 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 57/112] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:37.291008 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ebeb4-77ad-47b5-b8d7-f2a27beb8a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:37.292353 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ebeb4-77ad-47b5-b8d7-f2a27beb8a76 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:37.292635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ebeb4-77ad-47b5-b8d7-f2a27beb8a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:37.293113 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ebeb4-77ad-47b5-b8d7-f2a27beb8a76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:37.293805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ebeb4-77ad-47b5-b8d7-f2a27beb8a76 None admin] RBAC: Adding query filter params (role.id=967fbd121d5e4d65b135451eaf2c03d8, scope.project.id=52fe574d05694a42951535f0d8667f77, user.id=6aae10ffa9a84f9d901cc774261a3a09) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:37.294230 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ebeb4-77ad-47b5-b8d7-f2a27beb8a76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:37.295789 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ebeb4-77ad-47b5-b8d7-f2a27beb8a76 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:37.302092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 56/113] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:09:37 2022] GET /identity/v3/role_assignments?user.id=6aae10ffa9a84f9d901cc774261a3a09&scope.project.id=52fe574d05694a42951535f0d8667f77&role.id=967fbd121d5e4d65b135451eaf2c03d8 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:37.345631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c163b-ae82-4dc6-8347-7c53d9c1d582 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:37.441413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:37.490373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c163b-ae82-4dc6-8347-7c53d9c1d582 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:37.539224 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8272a5-1fc3-4c08-965c-68ddad6d754d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:37.539911 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8272a5-1fc3-4c08-965c-68ddad6d754d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:37.540193 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8272a5-1fc3-4c08-965c-68ddad6d754d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:37.540465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8272a5-1fc3-4c08-965c-68ddad6d754d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:37.551992 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8272a5-1fc3-4c08-965c-68ddad6d754d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:37.554585 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178bbd8-a16b-441c-b11a-1db6b6acd3dd None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:37.554871 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178bbd8-a16b-441c-b11a-1db6b6acd3dd None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:37.555145 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178bbd8-a16b-441c-b11a-1db6b6acd3dd None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:37.556537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 57/114] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:09:37.561403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-2e8272a5-1fc3-4c08-965c-68ddad6d754d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:09:37.561403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:37.561403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:37.561403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:37.561403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:37.561403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:37.561403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:37.561403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:37.561403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:37.561403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:37.561403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:37.561403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:37.561403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:37.561403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:37.561403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:37.561403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:37.562253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:37.562253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:37.562253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:37.562253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:37.562253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:37.562253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:37.562253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:37.562253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:37.562253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:37.562253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:37.562253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:37.562253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:37.562253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:37.562253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:37.562253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:37.562253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:37.563025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:37.563025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:37.563025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:37.563025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:37.563025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:37.563025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:37.563025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:37.563025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:37.563025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:37.563025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:37.563025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:09:37.563025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:09:37.563025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 58/115] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:37.568256 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b178bbd8-a16b-441c-b11a-1db6b6acd3dd None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:37.573250 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316a9bc-f518-4a6f-a4d7-2e48ba176bf9 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:37.573543 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316a9bc-f518-4a6f-a4d7-2e48ba176bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:37.573827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316a9bc-f518-4a6f-a4d7-2e48ba176bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:37.601499 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-7316a9bc-f518-4a6f-a4d7-2e48ba176bf9 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:37.603112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7316a9bc-f518-4a6f-a4d7-2e48ba176bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:37.608170 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:37.696140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b178bbd8-a16b-441c-b11a-1db6b6acd3dd None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:37.736738 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 59/116] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:37.738208 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e18ba-da7b-40a8-8ae1-d5383dcf26a8 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:37.738503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e18ba-da7b-40a8-8ae1-d5383dcf26a8 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:37.738780 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e18ba-da7b-40a8-8ae1-d5383dcf26a8 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:37.740072 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 60/117] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:37.746352 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e74998-791b-4697-a5d0-13bf244eb7f0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:37.746801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e74998-791b-4697-a5d0-13bf244eb7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:37.747090 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e74998-791b-4697-a5d0-13bf244eb7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:37.766438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-14e74998-791b-4697-a5d0-13bf244eb7f0 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:37.767649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-14e74998-791b-4697-a5d0-13bf244eb7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:37.775507 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c641d63-ee82-4127-b020-3eeba9dc6dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:37.777240 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c641d63-ee82-4127-b020-3eeba9dc6dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:37.777518 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c641d63-ee82-4127-b020-3eeba9dc6dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:37.777944 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c641d63-ee82-4127-b020-3eeba9dc6dc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:37.778997 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c641d63-ee82-4127-b020-3eeba9dc6dc0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:37.779303 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c641d63-ee82-4127-b020-3eeba9dc6dc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:37.781408 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c641d63-ee82-4127-b020-3eeba9dc6dc0 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:37.799980 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 58/118] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:09:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:37.800940 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c641d63-ee82-4127-b020-3eeba9dc6dc0 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:37.843982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:37.874188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 61/119] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:09:37.874877 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e74998-791b-4697-a5d0-13bf244eb7f0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:37.932442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:37.980805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c641d63-ee82-4127-b020-3eeba9dc6dc0 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:38.037731 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc25f89-0a7e-4c83-8a41-08265f8921e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:38.038793 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc25f89-0a7e-4c83-8a41-08265f8921e2 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:38.039273 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc25f89-0a7e-4c83-8a41-08265f8921e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:38.039754 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc25f89-0a7e-4c83-8a41-08265f8921e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:38.066874 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc25f89-0a7e-4c83-8a41-08265f8921e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:38.067493 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e74998-791b-4697-a5d0-13bf244eb7f0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:38.073777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-afc25f89-0a7e-4c83-8a41-08265f8921e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:38.073777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:38.073777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:38.073777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:38.073777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:38.073777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:38.073777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:38.073777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:38.073777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:38.073777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:38.073777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:38.073777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:38.073777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:38.073777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:38.073777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:38.073777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:38.074631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:38.074631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:38.074631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:38.074631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:38.074631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:38.074631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:38.074631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:38.074631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:38.074631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:38.074631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:38.074631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:38.074631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:38.074631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:38.074631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:38.074631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:38.074631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:38.075410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:38.075410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:38.075410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:38.075410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:38.075410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:38.075410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:38.075410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:38.075410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:38.075410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:38.075410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:38.075410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:38.075410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:09:38.091413 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 59/120] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:09:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:38.095440 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc25f89-0a7e-4c83-8a41-08265f8921e2 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:38.122686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aaab55-90de-4cf5-838d-4cfd45e16a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:38.123670 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aaab55-90de-4cf5-838d-4cfd45e16a95 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:38.125024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aaab55-90de-4cf5-838d-4cfd45e16a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:38.126197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aaab55-90de-4cf5-838d-4cfd45e16a95 None admin] PATH_INFO: `/v3/users/6aeda366bbb94f7cb91aa6788d9285e7` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:38.128784 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aaab55-90de-4cf5-838d-4cfd45e16a95 None admin] RBAC: Authorizing `identity:get_user(user_id=6aeda366bbb94f7cb91aa6788d9285e7)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:38.130676 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aaab55-90de-4cf5-838d-4cfd45e16a95 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:38.139972 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 62/121] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:37 2022] GET /identity/v3/users/6aeda366bbb94f7cb91aa6788d9285e7 => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:38.141053 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aaab55-90de-4cf5-838d-4cfd45e16a95 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:38.245078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:38.272471 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc25f89-0a7e-4c83-8a41-08265f8921e2 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:38.305928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aaab55-90de-4cf5-838d-4cfd45e16a95 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:38.314380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b401d-0940-483d-8016-5549f46e3c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:38.315642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b401d-0940-483d-8016-5549f46e3c44 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:38.316099 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b401d-0940-483d-8016-5549f46e3c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:38.316521 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b401d-0940-483d-8016-5549f46e3c44 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:38.317711 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b401d-0940-483d-8016-5549f46e3c44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:38.318509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b401d-0940-483d-8016-5549f46e3c44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:38.320384 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b401d-0940-483d-8016-5549f46e3c44 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:38.323048 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:38.332786 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 60/122] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:38.337395 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b401d-0940-483d-8016-5549f46e3c44 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:38.349574 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50ca32-2be2-4abe-b4eb-19c19f312054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:38.350976 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50ca32-2be2-4abe-b4eb-19c19f312054 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:38.351463 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50ca32-2be2-4abe-b4eb-19c19f312054 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:38.352355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50ca32-2be2-4abe-b4eb-19c19f312054 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:38.366764 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50ca32-2be2-4abe-b4eb-19c19f312054 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:38.374719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-8f50ca32-2be2-4abe-b4eb-19c19f312054 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:09:38.374719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:38.374719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:38.374719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:38.374719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:38.374719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:38.374719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:38.374719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:38.374719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:38.374719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:38.374719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:38.374719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:38.374719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:38.374719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:38.374719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:38.374719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:38.375581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:38.375581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:38.375581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:38.375581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:38.375581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:38.375581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:38.375581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:38.375581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:38.375581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:38.375581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:38.375581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:38.375581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:38.375581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:38.375581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:38.375581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:38.375581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:38.376517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:38.376517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:38.376517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:38.376517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:38.376517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:38.376517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:38.376517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:38.376517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:38.376517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:38.376517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:38.376517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:09:38.376517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:09:38.379463 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 63/123] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:38.380565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50ca32-2be2-4abe-b4eb-19c19f312054 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:38.459447 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b401d-0940-483d-8016-5549f46e3c44 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:38.508872 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13c747-bb20-4757-b540-c9e2ff82d3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:38.510671 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13c747-bb20-4757-b540-c9e2ff82d3ae None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:38.511109 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13c747-bb20-4757-b540-c9e2ff82d3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:38.511795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13c747-bb20-4757-b540-c9e2ff82d3ae None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:38.513753 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13c747-bb20-4757-b540-c9e2ff82d3ae None admin] RBAC: Authorizing `identity:get_project(project_id=52fe574d05694a42951535f0d8667f77)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:38.515384 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13c747-bb20-4757-b540-c9e2ff82d3ae None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:38.515725 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50ca32-2be2-4abe-b4eb-19c19f312054 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:38.517652 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 61/124] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:38 2022] GET /identity/v3/projects/52fe574d05694a42951535f0d8667f77 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:38.519637 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13c747-bb20-4757-b540-c9e2ff82d3ae None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:38.535305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:38.557932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8a3cf-c2fb-4c19-861e-f146384394b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:38.558570 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8a3cf-c2fb-4c19-861e-f146384394b0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:38.558850 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8a3cf-c2fb-4c19-861e-f146384394b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:38.559310 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8a3cf-c2fb-4c19-861e-f146384394b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:38.569697 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd8a3cf-c2fb-4c19-861e-f146384394b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:38.575187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-ddd8a3cf-c2fb-4c19-861e-f146384394b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:09:38.575187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:38.575187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:38.575187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:38.575187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:38.575187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:38.575187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:38.575187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:38.575187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:38.575187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:38.575187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:38.575187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:38.575187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:38.575187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:38.575187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:38.575187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:38.576305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:38.576305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:38.576305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:38.576305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:38.576305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:38.576305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:38.576305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:38.576305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:38.576305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:38.576305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:38.576305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:38.576305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:38.576305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:38.576305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:38.576305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:38.576305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:38.577240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:38.577240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:38.577240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:38.577240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:38.577240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:38.577240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:38.577240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:38.577240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:38.577240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:38.577240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:38.577240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:09:38.577240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:09:38.577240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 64/125] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:09:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:38.577240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8a3cf-c2fb-4c19-861e-f146384394b0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:38.611562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:38.659464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13c747-bb20-4757-b540-c9e2ff82d3ae None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:38.708474 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8a3cf-c2fb-4c19-861e-f146384394b0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:38.714346 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43048f-fa75-4a94-8d3f-6f779b9773f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:38.715254 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43048f-fa75-4a94-8d3f-6f779b9773f5 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:38.715793 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43048f-fa75-4a94-8d3f-6f779b9773f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:38.716767 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43048f-fa75-4a94-8d3f-6f779b9773f5 None admin] PATH_INFO: `/v3/users/dd316848081f4ae2ab8647785189b3f8` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:38.718604 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43048f-fa75-4a94-8d3f-6f779b9773f5 None admin] RBAC: Authorizing `identity:get_user(user_id=dd316848081f4ae2ab8647785189b3f8)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:38.721004 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43048f-fa75-4a94-8d3f-6f779b9773f5 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:38.724378 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 62/126] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:38 2022] GET /identity/v3/users/dd316848081f4ae2ab8647785189b3f8 => generated 247 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:38.727879 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43048f-fa75-4a94-8d3f-6f779b9773f5 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:38.762775 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d410a5-e1e0-4c40-ba9a-50af5aa72053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:38.763702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d410a5-e1e0-4c40-ba9a-50af5aa72053 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:38.764505 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d410a5-e1e0-4c40-ba9a-50af5aa72053 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:38.765009 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d410a5-e1e0-4c40-ba9a-50af5aa72053 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:38.767160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d410a5-e1e0-4c40-ba9a-50af5aa72053 None admin] RBAC: Adding query filter params (name=member) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:38.771545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d410a5-e1e0-4c40-ba9a-50af5aa72053 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:38.773633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d410a5-e1e0-4c40-ba9a-50af5aa72053 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:38.789733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 65/127] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:09:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:38.790658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d410a5-e1e0-4c40-ba9a-50af5aa72053 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:38.840167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:38.849767 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43048f-fa75-4a94-8d3f-6f779b9773f5 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:38.893528 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd62d9-0c32-49e3-8e8a-7be50f450f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:38.894768 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd62d9-0c32-49e3-8e8a-7be50f450f9b None admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:38.895462 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd62d9-0c32-49e3-8e8a-7be50f450f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:38.895923 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd62d9-0c32-49e3-8e8a-7be50f450f9b None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77/users/6aeda366bbb94f7cb91aa6788d9285e7/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:38.897289 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d410a5-e1e0-4c40-ba9a-50af5aa72053 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:38.898199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd62d9-0c32-49e3-8e8a-7be50f450f9b None admin] RBAC: Authorizing `identity:create_grant(project_id=52fe574d05694a42951535f0d8667f77, user_id=6aeda366bbb94f7cb91aa6788d9285e7, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:38.900880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd62d9-0c32-49e3-8e8a-7be50f450f9b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:38.903443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:38.920328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 63/128] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:09:38 2022] PUT /identity/v3/projects/52fe574d05694a42951535f0d8667f77/users/6aeda366bbb94f7cb91aa6788d9285e7/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:09:38.921535 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd62d9-0c32-49e3-8e8a-7be50f450f9b None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:38.938314 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1dc0a-fe60-4eb7-814a-f4ade371183f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:38.938954 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1dc0a-fe60-4eb7-814a-f4ade371183f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:38.939343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1dc0a-fe60-4eb7-814a-f4ade371183f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:38.939772 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1dc0a-fe60-4eb7-814a-f4ade371183f None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:38.940674 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1dc0a-fe60-4eb7-814a-f4ade371183f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:38.941079 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1dc0a-fe60-4eb7-814a-f4ade371183f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:38.942729 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1dc0a-fe60-4eb7-814a-f4ade371183f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:38.953723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 66/129] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:09:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:38.954632 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1dc0a-fe60-4eb7-814a-f4ade371183f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:39.053853 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd62d9-0c32-49e3-8e8a-7be50f450f9b None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:39.095853 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53889b30-eac2-4dda-93f3-300b45cb2a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:39.097192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53889b30-eac2-4dda-93f3-300b45cb2a78 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:39.098162 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53889b30-eac2-4dda-93f3-300b45cb2a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:39.098595 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53889b30-eac2-4dda-93f3-300b45cb2a78 None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:39.100292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53889b30-eac2-4dda-93f3-300b45cb2a78 None admin] RBAC: Authorizing `identity:get_project(project_id=52fe574d05694a42951535f0d8667f77)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:39.102189 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53889b30-eac2-4dda-93f3-300b45cb2a78 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:39.104552 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 64/130] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:38 2022] GET /identity/v3/projects/52fe574d05694a42951535f0d8667f77 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:39.105594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53889b30-eac2-4dda-93f3-300b45cb2a78 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:39.140155 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:39.143113 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1dc0a-fe60-4eb7-814a-f4ade371183f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:39.179546 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:39.224421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f83ffb-3ad6-4dcd-a05c-98b2a55e1516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:39.225138 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f83ffb-3ad6-4dcd-a05c-98b2a55e1516 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:39.225425 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f83ffb-3ad6-4dcd-a05c-98b2a55e1516 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:39.225702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f83ffb-3ad6-4dcd-a05c-98b2a55e1516 None admin] PATH_INFO: `/v3/users/6aeda366bbb94f7cb91aa6788d9285e7` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:39.228329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f83ffb-3ad6-4dcd-a05c-98b2a55e1516 None admin] RBAC: Authorizing `identity:get_user(user_id=6aeda366bbb94f7cb91aa6788d9285e7)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:39.229885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f83ffb-3ad6-4dcd-a05c-98b2a55e1516 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:39.233820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 67/131] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:38 2022] GET /identity/v3/users/6aeda366bbb94f7cb91aa6788d9285e7 => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:39.234595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f83ffb-3ad6-4dcd-a05c-98b2a55e1516 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:39.253860 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53889b30-eac2-4dda-93f3-300b45cb2a78 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:39.296935 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca14393-da74-44d5-a185-8d0c0b5556a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:39.297739 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca14393-da74-44d5-a185-8d0c0b5556a8 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:39.298030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca14393-da74-44d5-a185-8d0c0b5556a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:39.298306 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca14393-da74-44d5-a185-8d0c0b5556a8 None admin] PATH_INFO: `/v3/users/6aeda366bbb94f7cb91aa6788d9285e7` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:39.299565 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca14393-da74-44d5-a185-8d0c0b5556a8 None admin] RBAC: Authorizing `identity:get_user(user_id=6aeda366bbb94f7cb91aa6788d9285e7)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:39.301135 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca14393-da74-44d5-a185-8d0c0b5556a8 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:39.302833 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 65/132] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:39 2022] GET /identity/v3/users/6aeda366bbb94f7cb91aa6788d9285e7 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:39.303602 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca14393-da74-44d5-a185-8d0c0b5556a8 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:39.373369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f83ffb-3ad6-4dcd-a05c-98b2a55e1516 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:39.421896 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986d86f-60cb-4453-8aaf-027a73003a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:39.422987 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986d86f-60cb-4453-8aaf-027a73003a07 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:39.423548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986d86f-60cb-4453-8aaf-027a73003a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:39.423861 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986d86f-60cb-4453-8aaf-027a73003a07 None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77/users/dd316848081f4ae2ab8647785189b3f8/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:39.426805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7986d86f-60cb-4453-8aaf-027a73003a07 None admin] RBAC: Authorizing `identity:create_grant(project_id=52fe574d05694a42951535f0d8667f77, user_id=dd316848081f4ae2ab8647785189b3f8, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:39.428161 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca14393-da74-44d5-a185-8d0c0b5556a8 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:39.428365 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7986d86f-60cb-4453-8aaf-027a73003a07 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:39.446804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 68/133] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:09:39 2022] PUT /identity/v3/projects/52fe574d05694a42951535f0d8667f77/users/dd316848081f4ae2ab8647785189b3f8/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:09:39.448399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986d86f-60cb-4453-8aaf-027a73003a07 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:39.465232 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:39.472239 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664e5f6-e99e-47df-a1b9-85001d991b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:39.473068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664e5f6-e99e-47df-a1b9-85001d991b01 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:39.473469 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664e5f6-e99e-47df-a1b9-85001d991b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:39.473890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664e5f6-e99e-47df-a1b9-85001d991b01 None admin] PATH_INFO: `/v3/projects/81a6489a782a48058c995b5765cdc625` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:39.475341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664e5f6-e99e-47df-a1b9-85001d991b01 None admin] RBAC: Authorizing `identity:get_project(project_id=81a6489a782a48058c995b5765cdc625)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:39.477698 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664e5f6-e99e-47df-a1b9-85001d991b01 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:39.480685 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 66/134] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:39 2022] GET /identity/v3/projects/81a6489a782a48058c995b5765cdc625 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:39.488182 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664e5f6-e99e-47df-a1b9-85001d991b01 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:39.493161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:39.656502 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986d86f-60cb-4453-8aaf-027a73003a07 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:39.721667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664e5f6-e99e-47df-a1b9-85001d991b01 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:39.721841 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1f229-9fd0-4390-a5b9-df2e04f65ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:39.722412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1f229-9fd0-4390-a5b9-df2e04f65ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:39.722757 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1f229-9fd0-4390-a5b9-df2e04f65ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:39.723049 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1f229-9fd0-4390-a5b9-df2e04f65ff9 None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:39.725508 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1f229-9fd0-4390-a5b9-df2e04f65ff9 None admin] RBAC: Authorizing `identity:get_project(project_id=52fe574d05694a42951535f0d8667f77)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:39.727294 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1f229-9fd0-4390-a5b9-df2e04f65ff9 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:39.729785 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 69/135] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:39 2022] GET /identity/v3/projects/52fe574d05694a42951535f0d8667f77 => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:39.731116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e31096-cbb1-4f76-83f6-e4615b487fa0 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:39.731518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e31096-cbb1-4f76-83f6-e4615b487fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:39.736147 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e31096-cbb1-4f76-83f6-e4615b487fa0 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:39.738110 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 70/136] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:39.739972 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e31096-cbb1-4f76-83f6-e4615b487fa0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:39.784236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e730663-0e67-4c0e-8a77-dded25a1aa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:39.784915 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e730663-0e67-4c0e-8a77-dded25a1aa2d None admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:39.785209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e730663-0e67-4c0e-8a77-dded25a1aa2d None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:39.785493 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e730663-0e67-4c0e-8a77-dded25a1aa2d None admin] PATH_INFO: `/v3/projects/81a6489a782a48058c995b5765cdc625/users/6aeda366bbb94f7cb91aa6788d9285e7/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:39.787670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e730663-0e67-4c0e-8a77-dded25a1aa2d None admin] RBAC: Authorizing `identity:create_grant(project_id=81a6489a782a48058c995b5765cdc625, user_id=6aeda366bbb94f7cb91aa6788d9285e7, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:39.789176 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e730663-0e67-4c0e-8a77-dded25a1aa2d None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:39.804777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 67/137] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:09:39 2022] PUT /identity/v3/projects/81a6489a782a48058c995b5765cdc625/users/6aeda366bbb94f7cb91aa6788d9285e7/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:09:39.806373 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba073009-50b0-4e00-92e1-f24d80f76441 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:39.806673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba073009-50b0-4e00-92e1-f24d80f76441 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:39.807561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba073009-50b0-4e00-92e1-f24d80f76441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:39.828613 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-ba073009-50b0-4e00-92e1-f24d80f76441 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:39.829873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ba073009-50b0-4e00-92e1-f24d80f76441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:39.875365 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:39.893660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e31096-cbb1-4f76-83f6-e4615b487fa0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:39.894342 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:39.909281 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 68/138] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:39.951998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d01d9-eb35-4d43-9745-189b5dabef2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:39.952989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d01d9-eb35-4d43-9745-189b5dabef2a None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:39.953278 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d01d9-eb35-4d43-9745-189b5dabef2a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:39.953568 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d01d9-eb35-4d43-9745-189b5dabef2a None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77/users/6aeda366bbb94f7cb91aa6788d9285e7/roles/4cc230d31ac94f4c89bf827491c01bd7` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:39.961973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d01d9-eb35-4d43-9745-189b5dabef2a None admin] RBAC: Authorizing `identity:create_grant(project_id=52fe574d05694a42951535f0d8667f77, user_id=6aeda366bbb94f7cb91aa6788d9285e7, role_id=4cc230d31ac94f4c89bf827491c01bd7)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:39.963883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d01d9-eb35-4d43-9745-189b5dabef2a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:39.988324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 71/139] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:09:39 2022] PUT /identity/v3/projects/52fe574d05694a42951535f0d8667f77/users/6aeda366bbb94f7cb91aa6788d9285e7/roles/4cc230d31ac94f4c89bf827491c01bd7 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:09:40.053892 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b2eec-5bfc-48ce-b4a4-a9ffccf4183a None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:40.054208 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b2eec-5bfc-48ce-b4a4-a9ffccf4183a None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:40.054462 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b2eec-5bfc-48ce-b4a4-a9ffccf4183a None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:40.056271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 69/140] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:40.070962 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23686463-9c02-4dc7-9381-abc66d9054af None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:40.071584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23686463-9c02-4dc7-9381-abc66d9054af None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:40.072079 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23686463-9c02-4dc7-9381-abc66d9054af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:40.098575 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-23686463-9c02-4dc7-9381-abc66d9054af None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:40.100204 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-23686463-9c02-4dc7-9381-abc66d9054af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:40.199164 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 72/141] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:40.264600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b2eec-5bfc-48ce-b4a4-a9ffccf4183a None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:40.377017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b2eec-5bfc-48ce-b4a4-a9ffccf4183a None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:40.412003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0f9f5-db8c-41f5-b397-28d1c0e087a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:40.412639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0f9f5-db8c-41f5-b397-28d1c0e087a0 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:40.412922 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0f9f5-db8c-41f5-b397-28d1c0e087a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:40.413202 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0f9f5-db8c-41f5-b397-28d1c0e087a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:40.414332 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc0f9f5-db8c-41f5-b397-28d1c0e087a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:40.415846 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc0f9f5-db8c-41f5-b397-28d1c0e087a0 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:40.417962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 70/142] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:40.423383 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23686463-9c02-4dc7-9381-abc66d9054af None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:40.550765 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:40.569948 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23686463-9c02-4dc7-9381-abc66d9054af None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:40.612541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efeca81-071f-4c16-9e03-29bb8c7f87d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:40.613153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efeca81-071f-4c16-9e03-29bb8c7f87d3 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:40.613424 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efeca81-071f-4c16-9e03-29bb8c7f87d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:40.613693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efeca81-071f-4c16-9e03-29bb8c7f87d3 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:40.614362 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efeca81-071f-4c16-9e03-29bb8c7f87d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:40.615841 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efeca81-071f-4c16-9e03-29bb8c7f87d3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:40.671372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 73/143] 10.222.0.81 () {60 vars in 1132 bytes} [Mon Sep 19 11:09:40 2022] POST /identity/v3/users => generated 283 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:09:40.683578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0f9f5-db8c-41f5-b397-28d1c0e087a0 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:40.784344 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:40.876015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0f9f5-db8c-41f5-b397-28d1c0e087a0 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:40.941416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46943658-0c65-4d1c-8496-c5587cfb92d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:40.942361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46943658-0c65-4d1c-8496-c5587cfb92d8 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:40.942833 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46943658-0c65-4d1c-8496-c5587cfb92d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:40.943290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46943658-0c65-4d1c-8496-c5587cfb92d8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:40.964481 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46943658-0c65-4d1c-8496-c5587cfb92d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:40.969949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-46943658-0c65-4d1c-8496-c5587cfb92d8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:09:40.969949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:40.969949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:40.969949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:40.969949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:40.969949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:40.969949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:40.969949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:40.969949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:40.969949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:40.969949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:40.969949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:40.969949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:40.969949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:40.969949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:40.969949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:40.970863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:40.970863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:40.970863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:40.970863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:40.970863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:40.970863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:40.970863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:40.970863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:40.970863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:40.970863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:40.970863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:40.970863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:40.970863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:40.970863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:40.970863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:40.970863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:40.971642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:40.971642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:40.971642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:40.971642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:40.971642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:40.971642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:40.971642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:40.971642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:40.971642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:40.971642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:40.971642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:09:40.971642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:09:40.973721 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 71/144] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:40.984932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efeca81-071f-4c16-9e03-29bb8c7f87d3 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:41.121297 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efeca81-071f-4c16-9e03-29bb8c7f87d3 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:41.162097 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea455f6-f532-416c-9c08-b9e031ea60d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:41.162635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea455f6-f532-416c-9c08-b9e031ea60d1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:41.162916 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea455f6-f532-416c-9c08-b9e031ea60d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:41.163189 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea455f6-f532-416c-9c08-b9e031ea60d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:41.163925 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea455f6-f532-416c-9c08-b9e031ea60d1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:41.164230 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea455f6-f532-416c-9c08-b9e031ea60d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:41.165629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea455f6-f532-416c-9c08-b9e031ea60d1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:41.176617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 74/145] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:09:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:41.182318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46943658-0c65-4d1c-8496-c5587cfb92d8 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:41.327625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46943658-0c65-4d1c-8496-c5587cfb92d8 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:41.351794 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:41.381050 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881991e8-b30f-4adf-a1b1-694e28f8790e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:41.381951 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881991e8-b30f-4adf-a1b1-694e28f8790e None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:41.382391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881991e8-b30f-4adf-a1b1-694e28f8790e None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:41.382820 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881991e8-b30f-4adf-a1b1-694e28f8790e None admin] PATH_INFO: `/v3/users/6aae10ffa9a84f9d901cc774261a3a09` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:41.384178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881991e8-b30f-4adf-a1b1-694e28f8790e None admin] RBAC: Authorizing `identity:get_user(user_id=6aae10ffa9a84f9d901cc774261a3a09)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:41.385831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881991e8-b30f-4adf-a1b1-694e28f8790e None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:41.387566 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 72/146] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:41 2022] GET /identity/v3/users/6aae10ffa9a84f9d901cc774261a3a09 => generated 289 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:41.396463 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea455f6-f532-416c-9c08-b9e031ea60d1 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:41.404919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:41.539963 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea455f6-f532-416c-9c08-b9e031ea60d1 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:41.581893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d178025-056f-49a4-95fb-782912b6e0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:41.582530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d178025-056f-49a4-95fb-782912b6e0bb None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:41.582814 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d178025-056f-49a4-95fb-782912b6e0bb None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:41.583093 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d178025-056f-49a4-95fb-782912b6e0bb None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:41.589011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d178025-056f-49a4-95fb-782912b6e0bb None admin] RBAC: Authorizing `identity:get_project(project_id=52fe574d05694a42951535f0d8667f77)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:41.590499 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d178025-056f-49a4-95fb-782912b6e0bb None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:41.592194 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 75/147] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:41 2022] GET /identity/v3/projects/52fe574d05694a42951535f0d8667f77 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:41.598019 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881991e8-b30f-4adf-a1b1-694e28f8790e None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:41.733490 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881991e8-b30f-4adf-a1b1-694e28f8790e None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:41.782164 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63102663-aeb6-42d0-9808-b99029b3a14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:41.782832 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63102663-aeb6-42d0-9808-b99029b3a14e None admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:41.783116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63102663-aeb6-42d0-9808-b99029b3a14e None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:41.783394 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63102663-aeb6-42d0-9808-b99029b3a14e None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77/users/6aae10ffa9a84f9d901cc774261a3a09/roles/967fbd121d5e4d65b135451eaf2c03d8` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:41.785952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63102663-aeb6-42d0-9808-b99029b3a14e None admin] RBAC: Authorizing `identity:create_grant(project_id=52fe574d05694a42951535f0d8667f77, user_id=6aae10ffa9a84f9d901cc774261a3a09, role_id=967fbd121d5e4d65b135451eaf2c03d8)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:41.787415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63102663-aeb6-42d0-9808-b99029b3a14e None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:41.823242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 73/148] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:09:41 2022] PUT /identity/v3/projects/52fe574d05694a42951535f0d8667f77/users/6aae10ffa9a84f9d901cc774261a3a09/roles/967fbd121d5e4d65b135451eaf2c03d8 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:09:42.696028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc34f1-97ed-42ab-a763-324aa023598a None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:42.697037 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc34f1-97ed-42ab-a763-324aa023598a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:42.698105 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc34f1-97ed-42ab-a763-324aa023598a None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:42.699716 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 76/149] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:42.707793 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d27cc-c816-4e34-9475-87ba5d31f93d None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:42.708752 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d27cc-c816-4e34-9475-87ba5d31f93d None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:42.709255 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d27cc-c816-4e34-9475-87ba5d31f93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:42.736946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-079d27cc-c816-4e34-9475-87ba5d31f93d None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:42.738535 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-079d27cc-c816-4e34-9475-87ba5d31f93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:42.794949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:42.837643 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 74/150] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:09:42.925813 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dad978-21b3-4cf3-82e3-5b42b2e771ea None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:42.926440 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dad978-21b3-4cf3-82e3-5b42b2e771ea None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:42.926928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dad978-21b3-4cf3-82e3-5b42b2e771ea None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:42.928869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 77/151] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:42.935343 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7081e0-2411-46d2-9279-cc6468df48ab None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:42.935940 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7081e0-2411-46d2-9279-cc6468df48ab None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:42.936428 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7081e0-2411-46d2-9279-cc6468df48ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:42.964873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-ea7081e0-2411-46d2-9279-cc6468df48ab None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:42.966462 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ea7081e0-2411-46d2-9279-cc6468df48ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:43.050201 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 75/152] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:43.252093 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256871a9-2743-4b12-a1bd-50956f6a1c16 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:43.252748 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256871a9-2743-4b12-a1bd-50956f6a1c16 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:43.253251 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256871a9-2743-4b12-a1bd-50956f6a1c16 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:43.254803 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 78/153] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:43.260732 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ba619-c71c-492e-923b-075c58bf4e75 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:43.261304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ba619-c71c-492e-923b-075c58bf4e75 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:43.261823 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ba619-c71c-492e-923b-075c58bf4e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:43.289032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-7f3ba619-c71c-492e-923b-075c58bf4e75 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:43.290591 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-7f3ba619-c71c-492e-923b-075c58bf4e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:43.383187 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 76/154] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:43.537081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256871a9-2743-4b12-a1bd-50956f6a1c16 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:43.556464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d488ae1-585a-4c03-abe2-c596c3131f93 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:43.557723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d488ae1-585a-4c03-abe2-c596c3131f93 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:43.558226 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d488ae1-585a-4c03-abe2-c596c3131f93 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:43.561138 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 77/155] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:43.562074 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d488ae1-585a-4c03-abe2-c596c3131f93 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:43.623616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:43.680489 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:43.748523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d488ae1-585a-4c03-abe2-c596c3131f93 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:43.757032 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256871a9-2743-4b12-a1bd-50956f6a1c16 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:43.812924 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd19e2e-a218-40e9-b822-c5479243dc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:43.813477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd19e2e-a218-40e9-b822-c5479243dc4c None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:43.813759 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd19e2e-a218-40e9-b822-c5479243dc4c None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:43.814031 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd19e2e-a218-40e9-b822-c5479243dc4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:43.822097 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b1a29-cf35-4330-9430-5035de0ea093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:43.823294 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b1a29-cf35-4330-9430-5035de0ea093 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:43.823576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b1a29-cf35-4330-9430-5035de0ea093 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:43.823878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b1a29-cf35-4330-9430-5035de0ea093 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:43.835139 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd19e2e-a218-40e9-b822-c5479243dc4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:43.840773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-7bd19e2e-a218-40e9-b822-c5479243dc4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:09:43.840773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:43.840773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:43.840773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:43.840773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:43.840773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:43.840773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:43.840773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:43.840773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:43.840773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:43.840773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:43.840773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:43.840773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:43.840773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:43.840773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:43.840773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:43.841755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:43.841755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:43.841755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:43.841755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:43.841755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:43.841755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:43.841755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:43.841755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:43.841755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:43.841755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:43.841755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:43.841755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:43.841755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:43.841755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:43.841755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:43.841755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:43.842562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:43.842562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:43.842562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:43.842562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:43.842562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:43.842562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:43.842562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:43.842562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:43.842562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:43.842562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:43.842562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:09:43.842562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:09:43.845001 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 78/156] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:43.846171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b1a29-cf35-4330-9430-5035de0ea093 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:43.847323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bae3e-b4e1-4238-ae15-086163e3e7c9 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:43.847865 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bae3e-b4e1-4238-ae15-086163e3e7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:43.848349 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bae3e-b4e1-4238-ae15-086163e3e7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:43.851756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-b29b1a29-cf35-4330-9430-5035de0ea093 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:09:43.851756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:43.851756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:43.851756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:43.851756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:43.851756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:43.851756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:43.851756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:43.851756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:43.851756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:43.851756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:43.851756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:43.851756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:43.851756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:43.851756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:43.851756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:43.852825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:43.852825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:43.852825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:43.852825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:43.852825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:43.852825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:43.852825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:43.852825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:43.852825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:43.852825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:43.852825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:43.852825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:43.852825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:43.852825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:43.852825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:43.852825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:43.853720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:43.853720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:43.853720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:43.853720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:43.853720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:43.853720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:43.853720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:43.853720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:43.853720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:43.853720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:43.853720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:09:43.853720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:09:43.860274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 79/157] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:09:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:43.862055 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b1a29-cf35-4330-9430-5035de0ea093 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:43.888880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-598bae3e-b4e1-4238-ae15-086163e3e7c9 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:43.890234 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-598bae3e-b4e1-4238-ae15-086163e3e7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:43.994640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b1a29-cf35-4330-9430-5035de0ea093 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:44.005218 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 79/158] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:44.005990 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bae3e-b4e1-4238-ae15-086163e3e7c9 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:44.044445 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:44.058698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3800a-94f9-4295-abe6-f9ddc1011e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:44.061460 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3800a-94f9-4295-abe6-f9ddc1011e72 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:44.061937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3800a-94f9-4295-abe6-f9ddc1011e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:44.062393 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3800a-94f9-4295-abe6-f9ddc1011e72 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:44.068018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:44.068455 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3800a-94f9-4295-abe6-f9ddc1011e72 None admin] RBAC: Adding query filter params (name=member) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:44.068971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3800a-94f9-4295-abe6-f9ddc1011e72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:44.070623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3800a-94f9-4295-abe6-f9ddc1011e72 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:44.087195 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 80/159] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:09:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:44.089550 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3800a-94f9-4295-abe6-f9ddc1011e72 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:44.191039 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bae3e-b4e1-4238-ae15-086163e3e7c9 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:44.242756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3800a-94f9-4295-abe6-f9ddc1011e72 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:44.263094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6ce27-cb88-4eb6-967b-4337dce8c2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:44.265159 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6ce27-cb88-4eb6-967b-4337dce8c2ec None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:44.265638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6ce27-cb88-4eb6-967b-4337dce8c2ec None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:44.266071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6ce27-cb88-4eb6-967b-4337dce8c2ec None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:44.268225 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6ce27-cb88-4eb6-967b-4337dce8c2ec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:44.268709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6ce27-cb88-4eb6-967b-4337dce8c2ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:44.270251 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6ce27-cb88-4eb6-967b-4337dce8c2ec None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:44.293055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 80/160] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:09:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:44.294147 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6ce27-cb88-4eb6-967b-4337dce8c2ec None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:44.353095 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd823bc4-ad00-4fbe-9fb2-fd168b3b9f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:44.353912 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd823bc4-ad00-4fbe-9fb2-fd168b3b9f0a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:44.354167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd823bc4-ad00-4fbe-9fb2-fd168b3b9f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:44.354442 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd823bc4-ad00-4fbe-9fb2-fd168b3b9f0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:44.401299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd823bc4-ad00-4fbe-9fb2-fd168b3b9f0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:44.406200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-dd823bc4-ad00-4fbe-9fb2-fd168b3b9f0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:44.406200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:44.406200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:44.406200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:44.406200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:44.406200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:44.406200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:44.406200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:44.406200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:44.406200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:44.406200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:44.406200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:44.406200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:44.406200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:44.406200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:44.406200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:44.407133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:44.407133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:44.407133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:44.407133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:44.407133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:44.407133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:44.407133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:44.407133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:44.407133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:44.407133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:44.407133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:44.407133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:44.407133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:44.407133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:44.407133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:44.407133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:44.407940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:44.407940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:44.407940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:44.407940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:44.407940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:44.407940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:44.407940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:44.407940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:44.407940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:44.407940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:44.407940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:44.407940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:09:44.407940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 81/161] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:09:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:44.412726 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd823bc4-ad00-4fbe-9fb2-fd168b3b9f0a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:44.431380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:44.462902 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6ce27-cb88-4eb6-967b-4337dce8c2ec None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:44.519925 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:44.536483 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaaba94-2d21-4a6e-ac8a-813482312e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:44.537406 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaaba94-2d21-4a6e-ac8a-813482312e29 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:44.537691 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaaba94-2d21-4a6e-ac8a-813482312e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:44.537971 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaaba94-2d21-4a6e-ac8a-813482312e29 None admin] PATH_INFO: `/v3/users/6aeda366bbb94f7cb91aa6788d9285e7` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:44.539173 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaaba94-2d21-4a6e-ac8a-813482312e29 None admin] RBAC: Authorizing `identity:get_user(user_id=6aeda366bbb94f7cb91aa6788d9285e7)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:44.540762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaaba94-2d21-4a6e-ac8a-813482312e29 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:44.542533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 81/162] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:44 2022] GET /identity/v3/users/6aeda366bbb94f7cb91aa6788d9285e7 => generated 275 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:44.543306 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaaba94-2d21-4a6e-ac8a-813482312e29 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:44.638542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd823bc4-ad00-4fbe-9fb2-fd168b3b9f0a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:44.698779 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaaba94-2d21-4a6e-ac8a-813482312e29 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:44.760273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d9df8-3d9d-43e6-99ec-fd12f9121376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:44.767783 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d9df8-3d9d-43e6-99ec-fd12f9121376 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:44.768489 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d9df8-3d9d-43e6-99ec-fd12f9121376 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:44.768972 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d9df8-3d9d-43e6-99ec-fd12f9121376 None admin] PATH_INFO: `/v3/users/6aeda366bbb94f7cb91aa6788d9285e7` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:44.770393 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d9df8-3d9d-43e6-99ec-fd12f9121376 None admin] RBAC: Authorizing `identity:get_user(user_id=6aeda366bbb94f7cb91aa6788d9285e7)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:44.772104 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d9df8-3d9d-43e6-99ec-fd12f9121376 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:44.773986 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 82/163] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:44 2022] GET /identity/v3/users/6aeda366bbb94f7cb91aa6788d9285e7 => generated 275 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:44.774965 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d9df8-3d9d-43e6-99ec-fd12f9121376 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:44.784806 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70ba7f-5302-466c-986d-74543ee0f8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:44.785454 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70ba7f-5302-466c-986d-74543ee0f8ac None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:44.785733 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70ba7f-5302-466c-986d-74543ee0f8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:44.786004 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70ba7f-5302-466c-986d-74543ee0f8ac None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:44.786686 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70ba7f-5302-466c-986d-74543ee0f8ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:44.786992 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70ba7f-5302-466c-986d-74543ee0f8ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:44.789755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70ba7f-5302-466c-986d-74543ee0f8ac None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:44.829011 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 82/164] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:44.830152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70ba7f-5302-466c-986d-74543ee0f8ac None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:44.900191 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:45.061802 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d9df8-3d9d-43e6-99ec-fd12f9121376 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:45.111167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:45.136521 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a85715-1a6c-4111-b8df-d8df91ea73da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:45.137153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a85715-1a6c-4111-b8df-d8df91ea73da None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:45.137757 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a85715-1a6c-4111-b8df-d8df91ea73da None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:45.138037 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a85715-1a6c-4111-b8df-d8df91ea73da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:45.209883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70ba7f-5302-466c-986d-74543ee0f8ac None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:45.292554 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a85715-1a6c-4111-b8df-d8df91ea73da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:45.294206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a85715-1a6c-4111-b8df-d8df91ea73da None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:45.360446 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cbd42-578c-4bcd-a397-960564f0671d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:45.361484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cbd42-578c-4bcd-a397-960564f0671d None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:45.361965 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cbd42-578c-4bcd-a397-960564f0671d None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:45.362425 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cbd42-578c-4bcd-a397-960564f0671d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:45.404768 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cbd42-578c-4bcd-a397-960564f0671d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:45.411391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-f19cbd42-578c-4bcd-a397-960564f0671d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:09:45.411391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:45.411391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:45.411391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:45.411391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:45.411391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:45.411391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:45.411391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:45.411391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:45.411391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:45.411391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:45.411391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:45.411391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:45.411391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:45.411391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:45.411391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:45.412305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:45.412305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:45.412305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:45.412305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:45.412305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:45.412305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:45.412305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:45.412305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:45.412305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:45.412305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:45.412305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:45.412305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:45.412305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:45.412305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:45.412305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:45.412305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:45.413087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:45.413087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:45.413087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:45.413087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:45.413087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:45.413087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:45.413087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:45.413087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:45.413087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:45.413087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:45.413087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:09:45.413087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:09:45.415345 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 83/165] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 586 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:45.420930 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cbd42-578c-4bcd-a397-960564f0671d None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:45.529296 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 83/166] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 754 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:09:45.534786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a85715-1a6c-4111-b8df-d8df91ea73da None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:45.560199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:45.587540 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:45.634527 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cbd42-578c-4bcd-a397-960564f0671d None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:45.689365 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a85715-1a6c-4111-b8df-d8df91ea73da None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:45.732396 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f3f27-520c-43cd-8b03-8719618f65d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:45.733533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f3f27-520c-43cd-8b03-8719618f65d0 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:45.734469 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f3f27-520c-43cd-8b03-8719618f65d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:45.736321 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f3f27-520c-43cd-8b03-8719618f65d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:45.741212 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e60b0-1026-4606-b2dd-257b0385679e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:45.742244 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e60b0-1026-4606-b2dd-257b0385679e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:45.742835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e60b0-1026-4606-b2dd-257b0385679e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:45.745136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e60b0-1026-4606-b2dd-257b0385679e None admin] PATH_INFO: `/v3/users/dd316848081f4ae2ab8647785189b3f8` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:45.753135 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e60b0-1026-4606-b2dd-257b0385679e None admin] RBAC: Authorizing `identity:get_user(user_id=dd316848081f4ae2ab8647785189b3f8)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:45.755006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e60b0-1026-4606-b2dd-257b0385679e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:45.757064 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 84/167] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:45 2022] GET /identity/v3/users/dd316848081f4ae2ab8647785189b3f8 => generated 247 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:45.758214 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e60b0-1026-4606-b2dd-257b0385679e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:45.849606 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f3f27-520c-43cd-8b03-8719618f65d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:45.851794 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f3f27-520c-43cd-8b03-8719618f65d0 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:45.877775 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:45.888050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e60b0-1026-4606-b2dd-257b0385679e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:45.924464 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865012c1-c48f-46b6-8343-afe81a108420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:45.925070 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865012c1-c48f-46b6-8343-afe81a108420 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:45.925349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865012c1-c48f-46b6-8343-afe81a108420 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:45.925623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865012c1-c48f-46b6-8343-afe81a108420 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:45.926289 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865012c1-c48f-46b6-8343-afe81a108420 None admin] RBAC: Adding query filter params (name=member) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:45.926596 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865012c1-c48f-46b6-8343-afe81a108420 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:45.928011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865012c1-c48f-46b6-8343-afe81a108420 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:45.934028 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:45.937052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 85/168] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:09:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:45.942673 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a213f80-73e3-4f69-9911-5d35040a53ea None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:45.942959 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a213f80-73e3-4f69-9911-5d35040a53ea None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:45.943228 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a213f80-73e3-4f69-9911-5d35040a53ea None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:45.944521 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 86/169] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:45.945220 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a213f80-73e3-4f69-9911-5d35040a53ea None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:45.969352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 84/170] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:09:45.969352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f3f27-520c-43cd-8b03-8719618f65d0 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:46.058196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a213f80-73e3-4f69-9911-5d35040a53ea None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:46.074096 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f3f27-520c-43cd-8b03-8719618f65d0 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:46.092930 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4575a9-3128-4794-b2e3-85f7f0c0e966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:46.093560 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4575a9-3128-4794-b2e3-85f7f0c0e966 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:46.093839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4575a9-3128-4794-b2e3-85f7f0c0e966 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:46.094114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4575a9-3128-4794-b2e3-85f7f0c0e966 None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:46.095238 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4575a9-3128-4794-b2e3-85f7f0c0e966 None admin] RBAC: Authorizing `identity:get_project(project_id=52fe574d05694a42951535f0d8667f77)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:46.096730 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4575a9-3128-4794-b2e3-85f7f0c0e966 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:46.098635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 87/171] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:45 2022] GET /identity/v3/projects/52fe574d05694a42951535f0d8667f77 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:46.099549 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c41888-c8ef-41d2-9855-e0feb4518e16 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:46.099904 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c41888-c8ef-41d2-9855-e0feb4518e16 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:46.100127 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c41888-c8ef-41d2-9855-e0feb4518e16 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:46.101496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 88/172] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:46.106509 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f6390-0206-460b-9d7d-ec52e8d800f2 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:46.106780 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f6390-0206-460b-9d7d-ec52e8d800f2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:46.107055 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f6390-0206-460b-9d7d-ec52e8d800f2 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:46.111805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 89/173] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:46.111805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f6390-0206-460b-9d7d-ec52e8d800f2 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:46.117933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190c29a-4fe8-4422-b74c-c77f439be0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:46.118537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190c29a-4fe8-4422-b74c-c77f439be0ea None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:46.118812 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190c29a-4fe8-4422-b74c-c77f439be0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:46.119086 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190c29a-4fe8-4422-b74c-c77f439be0ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:46.131870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:46.165678 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:46.188902 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190c29a-4fe8-4422-b74c-c77f439be0ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:46.190365 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190c29a-4fe8-4422-b74c-c77f439be0ea None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:46.211319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f6390-0206-460b-9d7d-ec52e8d800f2 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:46.243727 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d41c8-76b2-43c4-854d-ccaa44a97713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:46.244389 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d41c8-76b2-43c4-854d-ccaa44a97713 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:46.244665 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d41c8-76b2-43c4-854d-ccaa44a97713 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:46.244939 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d41c8-76b2-43c4-854d-ccaa44a97713 None admin] PATH_INFO: `/v3/users/6aeda366bbb94f7cb91aa6788d9285e7` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:46.246059 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d41c8-76b2-43c4-854d-ccaa44a97713 None admin] RBAC: Authorizing `identity:get_user(user_id=6aeda366bbb94f7cb91aa6788d9285e7)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:46.247492 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d41c8-76b2-43c4-854d-ccaa44a97713 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:46.249088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 90/174] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:46 2022] GET /identity/v3/users/6aeda366bbb94f7cb91aa6788d9285e7 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:46.250159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614807a-5031-4d91-bef0-9696acb3d502 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:46.250438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614807a-5031-4d91-bef0-9696acb3d502 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:46.250877 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614807a-5031-4d91-bef0-9696acb3d502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:46.268682 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9614807a-5031-4d91-bef0-9696acb3d502 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:46.269858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9614807a-5031-4d91-bef0-9696acb3d502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:46.290264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 85/175] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:09:46.295195 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190c29a-4fe8-4422-b74c-c77f439be0ea None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:46.334031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 91/176] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:46.336684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614807a-5031-4d91-bef0-9696acb3d502 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:46.355171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:46.374042 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:46.402184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190c29a-4fe8-4422-b74c-c77f439be0ea None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:46.437213 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba2164-5341-42e8-8228-479d1d3bae15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:46.437831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba2164-5341-42e8-8228-479d1d3bae15 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:46.438109 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba2164-5341-42e8-8228-479d1d3bae15 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:46.438382 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba2164-5341-42e8-8228-479d1d3bae15 None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:46.439497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba2164-5341-42e8-8228-479d1d3bae15 None admin] RBAC: Authorizing `identity:get_project(project_id=52fe574d05694a42951535f0d8667f77)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:46.440976 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba2164-5341-42e8-8228-479d1d3bae15 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:46.441635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614807a-5031-4d91-bef0-9696acb3d502 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:46.442602 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 86/177] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:46 2022] GET /identity/v3/projects/52fe574d05694a42951535f0d8667f77 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:46.443750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca011a-06ef-48d0-8cc0-7f9b96dbfe70 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:46.444050 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca011a-06ef-48d0-8cc0-7f9b96dbfe70 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:46.444322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca011a-06ef-48d0-8cc0-7f9b96dbfe70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:46.462597 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-35ca011a-06ef-48d0-8cc0-7f9b96dbfe70 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:46.463857 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-35ca011a-06ef-48d0-8cc0-7f9b96dbfe70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:46.482639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960535fa-a9aa-4efb-9281-7863863cab89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:46.483483 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960535fa-a9aa-4efb-9281-7863863cab89 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:46.484094 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960535fa-a9aa-4efb-9281-7863863cab89 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:46.484438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960535fa-a9aa-4efb-9281-7863863cab89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:46.485235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960535fa-a9aa-4efb-9281-7863863cab89 None admin] RBAC: Adding query filter params (role.id=0c30c75e227c416da6e4814bce700e8d, scope.project.id=52fe574d05694a42951535f0d8667f77, user.id=6aeda366bbb94f7cb91aa6788d9285e7) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:46.485615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960535fa-a9aa-4efb-9281-7863863cab89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:46.487167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960535fa-a9aa-4efb-9281-7863863cab89 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:46.493435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 92/178] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:09:46 2022] GET /identity/v3/role_assignments?user.id=6aeda366bbb94f7cb91aa6788d9285e7&scope.project.id=52fe574d05694a42951535f0d8667f77&role.id=0c30c75e227c416da6e4814bce700e8d => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:46.498616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab2506-77d0-4201-a1be-e83b414a9dcc None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:46.498981 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab2506-77d0-4201-a1be-e83b414a9dcc None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:46.499233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab2506-77d0-4201-a1be-e83b414a9dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:46.523326 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f5ab2506-77d0-4201-a1be-e83b414a9dcc None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:46.525269 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f5ab2506-77d0-4201-a1be-e83b414a9dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:46.561064 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 87/179] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:46.563371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407b680-23f5-4a1e-b903-88a5d380f11b None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:46.564341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407b680-23f5-4a1e-b903-88a5d380f11b None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:46.565128 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407b680-23f5-4a1e-b903-88a5d380f11b None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:46.567693 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 88/180] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:46.568622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407b680-23f5-4a1e-b903-88a5d380f11b None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:46.627291 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 93/181] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:46.632267 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab2506-77d0-4201-a1be-e83b414a9dcc None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:46.661407 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:46.706135 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:46.739278 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407b680-23f5-4a1e-b903-88a5d380f11b None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:46.781235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933466bf-b60d-4438-a7a5-61b2322fd8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:46.781843 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933466bf-b60d-4438-a7a5-61b2322fd8c8 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:46.782119 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933466bf-b60d-4438-a7a5-61b2322fd8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:46.782389 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933466bf-b60d-4438-a7a5-61b2322fd8c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:46.782852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab2506-77d0-4201-a1be-e83b414a9dcc None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:46.831343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c0376-38a5-4ebd-a543-865b44794fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:46.832551 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c0376-38a5-4ebd-a543-865b44794fff None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:46.832900 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c0376-38a5-4ebd-a543-865b44794fff None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:46.833243 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c0376-38a5-4ebd-a543-865b44794fff None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:46.834816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c0376-38a5-4ebd-a543-865b44794fff None admin] RBAC: Authorizing `identity:get_project(project_id=52fe574d05694a42951535f0d8667f77)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:46.836700 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c0376-38a5-4ebd-a543-865b44794fff None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:46.840223 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 94/182] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:46 2022] GET /identity/v3/projects/52fe574d05694a42951535f0d8667f77 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:46.841339 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c0376-38a5-4ebd-a543-865b44794fff None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:46.869053 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933466bf-b60d-4438-a7a5-61b2322fd8c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:46.870568 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933466bf-b60d-4438-a7a5-61b2322fd8c8 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:46.949400 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:46.959674 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:46.969681 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c0376-38a5-4ebd-a543-865b44794fff None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:46.985000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 89/183] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:09:46.986206 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7d104-6dff-4b22-a77e-3ea676cce422 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:46.986488 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7d104-6dff-4b22-a77e-3ea676cce422 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:46.986758 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7d104-6dff-4b22-a77e-3ea676cce422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:47.003556 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-d1a7d104-6dff-4b22-a77e-3ea676cce422 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:47.004110 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd71075-2a57-4b99-95e0-ee7313c4041e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:47.004723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd71075-2a57-4b99-95e0-ee7313c4041e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:47.004820 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-d1a7d104-6dff-4b22-a77e-3ea676cce422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:47.005003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd71075-2a57-4b99-95e0-ee7313c4041e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:47.005272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd71075-2a57-4b99-95e0-ee7313c4041e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:47.006005 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd71075-2a57-4b99-95e0-ee7313c4041e None admin] RBAC: Adding query filter params (role.id=4cc230d31ac94f4c89bf827491c01bd7, scope.project.id=52fe574d05694a42951535f0d8667f77, user.id=6aeda366bbb94f7cb91aa6788d9285e7) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:47.006315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd71075-2a57-4b99-95e0-ee7313c4041e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:47.007754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd71075-2a57-4b99-95e0-ee7313c4041e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:47.014046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 95/184] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:09:46 2022] GET /identity/v3/role_assignments?user.id=6aeda366bbb94f7cb91aa6788d9285e7&scope.project.id=52fe574d05694a42951535f0d8667f77&role.id=4cc230d31ac94f4c89bf827491c01bd7 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:47.015244 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd71075-2a57-4b99-95e0-ee7313c4041e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:47.080950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 90/185] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:47.082197 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7d104-6dff-4b22-a77e-3ea676cce422 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:47.157276 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd71075-2a57-4b99-95e0-ee7313c4041e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:47.200655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a18bfd-7ecf-4c54-9e5e-98b3d1fb0058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:47.201217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a18bfd-7ecf-4c54-9e5e-98b3d1fb0058 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:47.201498 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a18bfd-7ecf-4c54-9e5e-98b3d1fb0058 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:47.201779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a18bfd-7ecf-4c54-9e5e-98b3d1fb0058 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:47.210363 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7d104-6dff-4b22-a77e-3ea676cce422 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:47.212841 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:47.219540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a18bfd-7ecf-4c54-9e5e-98b3d1fb0058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:47.225950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-d0a18bfd-7ecf-4c54-9e5e-98b3d1fb0058 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:47.225950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:47.225950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:47.225950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:47.225950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:47.225950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:47.225950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:47.225950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:47.225950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:47.225950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:47.225950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:47.225950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:47.225950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:47.225950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:47.225950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:47.225950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:47.226849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:47.226849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:47.226849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:47.226849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:47.226849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:47.226849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:47.226849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:47.226849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:47.226849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:47.226849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:47.226849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:47.226849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:47.226849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:47.226849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:47.226849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:47.226849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:47.229233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:47.229233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:47.229233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:47.229233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:47.229233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:47.229233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:47.229233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:47.229233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:47.229233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:47.229233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:47.229233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:47.229233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:09:47.229233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 96/186] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:09:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:47.229233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a18bfd-7ecf-4c54-9e5e-98b3d1fb0058 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:47.232793 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:47.272077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42890b7-f116-4aba-8241-4f36d13a6f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:47.273211 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42890b7-f116-4aba-8241-4f36d13a6f3b None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:47.273516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42890b7-f116-4aba-8241-4f36d13a6f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:47.273783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42890b7-f116-4aba-8241-4f36d13a6f3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:47.274807 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42890b7-f116-4aba-8241-4f36d13a6f3b None admin] RBAC: Adding query filter params (role.id=753f8b35d5f0458d8f62d0ad6e28033f, scope.project.id=52fe574d05694a42951535f0d8667f77, user.id=dd316848081f4ae2ab8647785189b3f8) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:47.275116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42890b7-f116-4aba-8241-4f36d13a6f3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:47.277162 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42890b7-f116-4aba-8241-4f36d13a6f3b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:47.283796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 91/187] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:09:47 2022] GET /identity/v3/role_assignments?user.id=dd316848081f4ae2ab8647785189b3f8&scope.project.id=52fe574d05694a42951535f0d8667f77&role.id=753f8b35d5f0458d8f62d0ad6e28033f => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:47.284411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42890b7-f116-4aba-8241-4f36d13a6f3b None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:47.378699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a18bfd-7ecf-4c54-9e5e-98b3d1fb0058 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:47.410858 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42890b7-f116-4aba-8241-4f36d13a6f3b None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:47.415857 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0ab6c-8e2f-480c-8b39-3a4f93cf2a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:47.418297 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0ab6c-8e2f-480c-8b39-3a4f93cf2a1c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:47.419186 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0ab6c-8e2f-480c-8b39-3a4f93cf2a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:47.419475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0ab6c-8e2f-480c-8b39-3a4f93cf2a1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:47.422586 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0ab6c-8e2f-480c-8b39-3a4f93cf2a1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:47.424738 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0ab6c-8e2f-480c-8b39-3a4f93cf2a1c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:47.428232 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 97/188] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:47.428232 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0ab6c-8e2f-480c-8b39-3a4f93cf2a1c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:47.451328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a8f1d-79db-4cdb-834e-ed3e15cfaba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:47.451935 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a8f1d-79db-4cdb-834e-ed3e15cfaba8 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:47.452217 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a8f1d-79db-4cdb-834e-ed3e15cfaba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:47.452488 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a8f1d-79db-4cdb-834e-ed3e15cfaba8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:47.465558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a8f1d-79db-4cdb-834e-ed3e15cfaba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:47.472160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-b45a8f1d-79db-4cdb-834e-ed3e15cfaba8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:09:47.472160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:47.472160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:47.472160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:47.472160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:47.472160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:47.472160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:47.472160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:47.472160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:47.472160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:47.472160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:47.472160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:47.472160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:47.472160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:47.472160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:47.472160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:47.473158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:47.473158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:47.473158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:47.473158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:47.473158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:47.473158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:47.473158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:47.473158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:47.473158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:47.473158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:47.473158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:47.473158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:47.473158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:47.473158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:47.473158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:47.473158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:47.473954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:47.473954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:47.473954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:47.473954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:47.473954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:47.473954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:47.473954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:47.473954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:47.473954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:47.473954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:47.473954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:09:47.473954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:09:47.473954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 92/189] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:09:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:47.474600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a8f1d-79db-4cdb-834e-ed3e15cfaba8 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:47.510336 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:47.512537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:47.549656 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0ab6c-8e2f-480c-8b39-3a4f93cf2a1c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:47.586379 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd8aab-8704-4eec-b987-1a98f0cc83f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:47.587014 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd8aab-8704-4eec-b987-1a98f0cc83f2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:47.587293 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd8aab-8704-4eec-b987-1a98f0cc83f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:47.587566 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd8aab-8704-4eec-b987-1a98f0cc83f2 None admin] PATH_INFO: `/v3/projects/81a6489a782a48058c995b5765cdc625` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:47.589278 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd8aab-8704-4eec-b987-1a98f0cc83f2 None admin] RBAC: Authorizing `identity:get_project(project_id=81a6489a782a48058c995b5765cdc625)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:47.590099 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a8f1d-79db-4cdb-834e-ed3e15cfaba8 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:47.590745 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd8aab-8704-4eec-b987-1a98f0cc83f2 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:47.592417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 98/190] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:47 2022] GET /identity/v3/projects/81a6489a782a48058c995b5765cdc625 => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:47.593173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd8aab-8704-4eec-b987-1a98f0cc83f2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:47.625169 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53671d9-4438-4975-bd64-4187510f2389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:47.625774 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53671d9-4438-4975-bd64-4187510f2389 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:47.626053 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53671d9-4438-4975-bd64-4187510f2389 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:47.626326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53671d9-4438-4975-bd64-4187510f2389 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:47.626990 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53671d9-4438-4975-bd64-4187510f2389 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:47.627295 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53671d9-4438-4975-bd64-4187510f2389 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:47.628692 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53671d9-4438-4975-bd64-4187510f2389 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:47.636719 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 93/191] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:47.637484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53671d9-4438-4975-bd64-4187510f2389 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:47.703603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd8aab-8704-4eec-b987-1a98f0cc83f2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:47.725919 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:47.736161 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53671d9-4438-4975-bd64-4187510f2389 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:47.736966 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44a9d5-c1da-4764-bd11-49c21018d154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:47.737594 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44a9d5-c1da-4764-bd11-49c21018d154 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:47.737870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44a9d5-c1da-4764-bd11-49c21018d154 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:47.738142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44a9d5-c1da-4764-bd11-49c21018d154 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:47.745854 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:47.750530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc44a9d5-c1da-4764-bd11-49c21018d154 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:47.755507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-bc44a9d5-c1da-4764-bd11-49c21018d154 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:09:47.755507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:47.755507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:47.755507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:47.755507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:47.755507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:47.755507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:47.755507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:47.755507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:47.755507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:47.755507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:47.755507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:47.755507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:47.755507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:47.755507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:47.755507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:47.756542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:47.756542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:47.756542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:47.756542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:47.756542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:47.756542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:47.756542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:47.756542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:47.756542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:47.756542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:47.756542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:47.756542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:47.756542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:47.756542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:47.756542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:47.756542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:47.757329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:47.757329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:47.757329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:47.757329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:47.757329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:47.757329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:47.757329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:47.757329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:47.757329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:47.757329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:47.757329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:09:47.757329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:09:47.757329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 99/192] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:47.757996 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44a9d5-c1da-4764-bd11-49c21018d154 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:47.774922 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26477cb4-77b5-42ba-aa83-36d9e8db58d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:47.775539 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26477cb4-77b5-42ba-aa83-36d9e8db58d9 None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:47.776357 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26477cb4-77b5-42ba-aa83-36d9e8db58d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:47.776629 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26477cb4-77b5-42ba-aa83-36d9e8db58d9 None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:47.777267 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26477cb4-77b5-42ba-aa83-36d9e8db58d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:47.778902 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26477cb4-77b5-42ba-aa83-36d9e8db58d9 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:47.805867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 94/193] 10.222.0.81 () {60 vars in 1132 bytes} [Mon Sep 19 11:09:47 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:09:47.805867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26477cb4-77b5-42ba-aa83-36d9e8db58d9 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:47.877572 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44a9d5-c1da-4764-bd11-49c21018d154 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:47.917910 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6442d4a-ad43-4b17-9a00-5e3d11192f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:47.918532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6442d4a-ad43-4b17-9a00-5e3d11192f08 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:47.918824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6442d4a-ad43-4b17-9a00-5e3d11192f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:47.919080 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6442d4a-ad43-4b17-9a00-5e3d11192f08 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:47.919792 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6442d4a-ad43-4b17-9a00-5e3d11192f08 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:47.920144 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6442d4a-ad43-4b17-9a00-5e3d11192f08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:47.921573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6442d4a-ad43-4b17-9a00-5e3d11192f08 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:47.923325 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26477cb4-77b5-42ba-aa83-36d9e8db58d9 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:47.930814 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 100/194] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:09:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:47.931530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6442d4a-ad43-4b17-9a00-5e3d11192f08 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:47.966530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000c5e9-8db0-4faf-bbaa-5fc441177d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:47.967073 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000c5e9-8db0-4faf-bbaa-5fc441177d7a None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:47.967349 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000c5e9-8db0-4faf-bbaa-5fc441177d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:47.967622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000c5e9-8db0-4faf-bbaa-5fc441177d7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:47.968399 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000c5e9-8db0-4faf-bbaa-5fc441177d7a None admin] RBAC: Adding query filter params (role.id=0c30c75e227c416da6e4814bce700e8d, scope.project.id=81a6489a782a48058c995b5765cdc625, user.id=6aeda366bbb94f7cb91aa6788d9285e7) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:47.968714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000c5e9-8db0-4faf-bbaa-5fc441177d7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:47.970178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000c5e9-8db0-4faf-bbaa-5fc441177d7a None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:47.976736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 95/195] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:09:47 2022] GET /identity/v3/role_assignments?user.id=6aeda366bbb94f7cb91aa6788d9285e7&scope.project.id=81a6489a782a48058c995b5765cdc625&role.id=0c30c75e227c416da6e4814bce700e8d => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:47.978449 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000c5e9-8db0-4faf-bbaa-5fc441177d7a None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:47.996440 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:48.016570 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:48.075345 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6442d4a-ad43-4b17-9a00-5e3d11192f08 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:48.121495 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaebbee-498e-4d43-91f4-9b743174ac0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:48.122128 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaebbee-498e-4d43-91f4-9b743174ac0c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:48.122414 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaebbee-498e-4d43-91f4-9b743174ac0c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:48.122703 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaebbee-498e-4d43-91f4-9b743174ac0c None admin] PATH_INFO: `/v3/users/aeaea0c2d8494dfab06bfd3933c13ab0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:48.125178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000c5e9-8db0-4faf-bbaa-5fc441177d7a None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:48.138831 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaebbee-498e-4d43-91f4-9b743174ac0c None admin] RBAC: Authorizing `identity:get_user(user_id=aeaea0c2d8494dfab06bfd3933c13ab0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:48.140381 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaebbee-498e-4d43-91f4-9b743174ac0c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:48.142299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 101/196] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:47 2022] GET /identity/v3/users/aeaea0c2d8494dfab06bfd3933c13ab0 => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:48.142990 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaebbee-498e-4d43-91f4-9b743174ac0c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:48.175069 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3275ba8-e3fc-4048-88c3-20db84943d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:48.175963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3275ba8-e3fc-4048-88c3-20db84943d2f None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:48.176238 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3275ba8-e3fc-4048-88c3-20db84943d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:48.176506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3275ba8-e3fc-4048-88c3-20db84943d2f None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:48.177172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3275ba8-e3fc-4048-88c3-20db84943d2f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:48.177477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3275ba8-e3fc-4048-88c3-20db84943d2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:48.178855 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3275ba8-e3fc-4048-88c3-20db84943d2f None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:48.193943 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 96/197] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:09:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:48.194794 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3275ba8-e3fc-4048-88c3-20db84943d2f None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:48.288399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaebbee-498e-4d43-91f4-9b743174ac0c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:48.306250 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:48.312628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:48.329393 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a0bcb-bc26-47c9-9d53-346b6f5eecc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:48.330165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a0bcb-bc26-47c9-9d53-346b6f5eecc9 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:48.330453 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a0bcb-bc26-47c9-9d53-346b6f5eecc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:48.330724 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a0bcb-bc26-47c9-9d53-346b6f5eecc9 None admin] PATH_INFO: `/v3/users/aeaea0c2d8494dfab06bfd3933c13ab0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:48.332353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a0bcb-bc26-47c9-9d53-346b6f5eecc9 None admin] RBAC: Authorizing `identity:get_user(user_id=aeaea0c2d8494dfab06bfd3933c13ab0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:48.333826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a0bcb-bc26-47c9-9d53-346b6f5eecc9 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:48.335588 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 102/198] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:48 2022] GET /identity/v3/users/aeaea0c2d8494dfab06bfd3933c13ab0 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:48.337279 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a0bcb-bc26-47c9-9d53-346b6f5eecc9 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:48.350834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3275ba8-e3fc-4048-88c3-20db84943d2f None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:48.387623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9a120-e583-4291-9ca6-83fcefd5abd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:48.388392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9a120-e583-4291-9ca6-83fcefd5abd5 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:48.388669 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9a120-e583-4291-9ca6-83fcefd5abd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:48.388935 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9a120-e583-4291-9ca6-83fcefd5abd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:48.466643 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a0bcb-bc26-47c9-9d53-346b6f5eecc9 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:48.470015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab9a120-e583-4291-9ca6-83fcefd5abd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:48.471493 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab9a120-e583-4291-9ca6-83fcefd5abd5 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:48.502771 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14826ea-be00-4778-9782-4983932a4526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:48.503440 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14826ea-be00-4778-9782-4983932a4526 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:48.503798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14826ea-be00-4778-9782-4983932a4526 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:48.504085 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14826ea-be00-4778-9782-4983932a4526 None admin] PATH_INFO: `/v3/users/6aae10ffa9a84f9d901cc774261a3a09` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:48.505332 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14826ea-be00-4778-9782-4983932a4526 None admin] RBAC: Authorizing `identity:get_user(user_id=6aae10ffa9a84f9d901cc774261a3a09)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:48.506971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14826ea-be00-4778-9782-4983932a4526 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:48.508825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 103/199] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:48 2022] GET /identity/v3/users/6aae10ffa9a84f9d901cc774261a3a09 => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:48.509596 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14826ea-be00-4778-9782-4983932a4526 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:48.558892 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:48.576844 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:48.603227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 97/200] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:09:48.605066 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9a120-e583-4291-9ca6-83fcefd5abd5 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:48.632035 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14826ea-be00-4778-9782-4983932a4526 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:48.679096 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b60852-98ad-4473-98bc-a26ab15be03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:48.680048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b60852-98ad-4473-98bc-a26ab15be03b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:48.680602 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b60852-98ad-4473-98bc-a26ab15be03b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:48.681014 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b60852-98ad-4473-98bc-a26ab15be03b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:48.725686 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9a120-e583-4291-9ca6-83fcefd5abd5 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:48.756637 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b60852-98ad-4473-98bc-a26ab15be03b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:48.758579 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b60852-98ad-4473-98bc-a26ab15be03b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:48.766497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e3806-5dc8-4918-ab1e-6df2d4cbf60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:48.767106 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e3806-5dc8-4918-ab1e-6df2d4cbf60d None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:48.767378 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e3806-5dc8-4918-ab1e-6df2d4cbf60d None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:48.767650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e3806-5dc8-4918-ab1e-6df2d4cbf60d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:48.814685 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:48.822816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:48.848936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e3806-5dc8-4918-ab1e-6df2d4cbf60d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:48.850855 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e3806-5dc8-4918-ab1e-6df2d4cbf60d None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:48.874094 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 104/201] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:09:48.875266 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b60852-98ad-4473-98bc-a26ab15be03b None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:48.975033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 98/202] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:09:48.976214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e3806-5dc8-4918-ab1e-6df2d4cbf60d None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:48.990304 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b60852-98ad-4473-98bc-a26ab15be03b None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:49.030338 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15c237-08ff-407d-9c07-6c00e63fc3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:49.031705 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15c237-08ff-407d-9c07-6c00e63fc3de None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:49.032150 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15c237-08ff-407d-9c07-6c00e63fc3de None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:49.032682 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15c237-08ff-407d-9c07-6c00e63fc3de None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:49.033980 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d15c237-08ff-407d-9c07-6c00e63fc3de None admin] RBAC: Authorizing `identity:get_project(project_id=8eb05b28107f4365a8f3ff9b192ce72f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:49.035588 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d15c237-08ff-407d-9c07-6c00e63fc3de None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:49.037495 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 105/203] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:48 2022] GET /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:49.038391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15c237-08ff-407d-9c07-6c00e63fc3de None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:49.060267 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:49.066768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:49.096573 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e3806-5dc8-4918-ab1e-6df2d4cbf60d None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:49.136382 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444e7cd-e0fb-41dd-a72d-860e1078499d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:49.137003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444e7cd-e0fb-41dd-a72d-860e1078499d None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:49.137306 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444e7cd-e0fb-41dd-a72d-860e1078499d None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:49.138047 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444e7cd-e0fb-41dd-a72d-860e1078499d None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:49.139454 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444e7cd-e0fb-41dd-a72d-860e1078499d None admin] RBAC: Authorizing `identity:get_project(project_id=8eb05b28107f4365a8f3ff9b192ce72f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:49.141044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444e7cd-e0fb-41dd-a72d-860e1078499d None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:49.142780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 99/204] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:48 2022] GET /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:49.143538 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444e7cd-e0fb-41dd-a72d-860e1078499d None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:49.151625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15c237-08ff-407d-9c07-6c00e63fc3de None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:49.190978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105d6bc-6616-4801-a81e-4d9ff36b48e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:49.191894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105d6bc-6616-4801-a81e-4d9ff36b48e4 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:49.192316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105d6bc-6616-4801-a81e-4d9ff36b48e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:49.192726 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105d6bc-6616-4801-a81e-4d9ff36b48e4 None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:49.193991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105d6bc-6616-4801-a81e-4d9ff36b48e4 None admin] RBAC: Authorizing `identity:get_project(project_id=52fe574d05694a42951535f0d8667f77)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:49.195595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105d6bc-6616-4801-a81e-4d9ff36b48e4 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:49.198007 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 106/205] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:49 2022] GET /identity/v3/projects/52fe574d05694a42951535f0d8667f77 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:49.198899 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105d6bc-6616-4801-a81e-4d9ff36b48e4 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:49.272292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444e7cd-e0fb-41dd-a72d-860e1078499d None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:49.305817 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8a555-b804-4545-8717-8e24f929c25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:49.306214 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:49.307906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8a555-b804-4545-8717-8e24f929c25d None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:49.308323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8a555-b804-4545-8717-8e24f929c25d None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:49.308728 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8a555-b804-4545-8717-8e24f929c25d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:49.309588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8a555-b804-4545-8717-8e24f929c25d None admin] RBAC: Adding query filter params (role.id=753f8b35d5f0458d8f62d0ad6e28033f, scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f, user.id=aeaea0c2d8494dfab06bfd3933c13ab0) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:49.310032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8a555-b804-4545-8717-8e24f929c25d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:49.311619 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8a555-b804-4545-8717-8e24f929c25d None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:49.318463 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 100/206] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:09:49 2022] GET /identity/v3/role_assignments?user.id=aeaea0c2d8494dfab06bfd3933c13ab0&scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f&role.id=753f8b35d5f0458d8f62d0ad6e28033f => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:49.319481 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8a555-b804-4545-8717-8e24f929c25d None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:49.321376 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105d6bc-6616-4801-a81e-4d9ff36b48e4 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:49.324307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:49.362434 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5784f5d-b352-4bb5-8038-250fd7222736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:49.363042 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5784f5d-b352-4bb5-8038-250fd7222736 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:49.363315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5784f5d-b352-4bb5-8038-250fd7222736 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:49.363584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5784f5d-b352-4bb5-8038-250fd7222736 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:49.364517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5784f5d-b352-4bb5-8038-250fd7222736 None admin] RBAC: Adding query filter params (role.id=4cc230d31ac94f4c89bf827491c01bd7, scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f, user.id=aeaea0c2d8494dfab06bfd3933c13ab0) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:49.364811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5784f5d-b352-4bb5-8038-250fd7222736 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:49.367034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5784f5d-b352-4bb5-8038-250fd7222736 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:49.378771 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 107/207] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:09:49 2022] GET /identity/v3/role_assignments?user.id=aeaea0c2d8494dfab06bfd3933c13ab0&scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f&role.id=4cc230d31ac94f4c89bf827491c01bd7 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:49.466249 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8a555-b804-4545-8717-8e24f929c25d None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:49.509645 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764e455-9b3b-47b0-b2a4-988a2d9d5d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:49.511227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764e455-9b3b-47b0-b2a4-988a2d9d5d30 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:49.511817 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764e455-9b3b-47b0-b2a4-988a2d9d5d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:49.512290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764e455-9b3b-47b0-b2a4-988a2d9d5d30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:49.513286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764e455-9b3b-47b0-b2a4-988a2d9d5d30 None admin] RBAC: Adding query filter params (role.id=967fbd121d5e4d65b135451eaf2c03d8, scope.project.id=52fe574d05694a42951535f0d8667f77, user.id=6aae10ffa9a84f9d901cc774261a3a09) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:49.513811 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764e455-9b3b-47b0-b2a4-988a2d9d5d30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:49.515584 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764e455-9b3b-47b0-b2a4-988a2d9d5d30 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:49.523168 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 101/208] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:09:49 2022] GET /identity/v3/role_assignments?user.id=6aae10ffa9a84f9d901cc774261a3a09&scope.project.id=52fe574d05694a42951535f0d8667f77&role.id=967fbd121d5e4d65b135451eaf2c03d8 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:50.276866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d910772-c282-467a-b43c-f64321970b5c None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:50.277887 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d910772-c282-467a-b43c-f64321970b5c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:50.278395 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d910772-c282-467a-b43c-f64321970b5c None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:50.279983 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 108/209] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:50.285685 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1da7ef-9120-4133-8a82-b52490d1b870 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:50.286634 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1da7ef-9120-4133-8a82-b52490d1b870 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:50.287110 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1da7ef-9120-4133-8a82-b52490d1b870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:50.310080 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-cc1da7ef-9120-4133-8a82-b52490d1b870 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:50.311850 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-cc1da7ef-9120-4133-8a82-b52490d1b870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:50.379752 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:50.386982 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 102/210] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:50.415493 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9edfe-a902-4fc5-9141-2d2b596b632a None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:50.416153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9edfe-a902-4fc5-9141-2d2b596b632a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:50.416706 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9edfe-a902-4fc5-9141-2d2b596b632a None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:50.418249 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 109/211] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:50.423561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d9fb4-18f2-4205-82eb-1b53ccb01389 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:50.424086 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d9fb4-18f2-4205-82eb-1b53ccb01389 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:50.424560 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d9fb4-18f2-4205-82eb-1b53ccb01389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:50.443540 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-401d9fb4-18f2-4205-82eb-1b53ccb01389 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:50.445030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-401d9fb4-18f2-4205-82eb-1b53ccb01389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:50.515507 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 103/212] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:50.658117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9edfe-a902-4fc5-9141-2d2b596b632a None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:50.771332 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9edfe-a902-4fc5-9141-2d2b596b632a None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:50.806003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4d984-41ae-4c47-ad8d-83b02f9a3147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:50.806637 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4d984-41ae-4c47-ad8d-83b02f9a3147 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:50.806946 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4d984-41ae-4c47-ad8d-83b02f9a3147 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:50.807219 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4d984-41ae-4c47-ad8d-83b02f9a3147 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:50.807824 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d9fb4-18f2-4205-82eb-1b53ccb01389 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:50.818941 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4d984-41ae-4c47-ad8d-83b02f9a3147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:50.824798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-e2c4d984-41ae-4c47-ad8d-83b02f9a3147 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:09:50.824798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:50.824798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:50.824798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:50.824798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:50.824798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:50.824798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:50.824798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:50.824798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:50.824798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:50.824798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:50.824798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:50.824798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:50.824798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:50.824798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:50.824798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:50.825677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:50.825677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:50.825677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:50.825677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:50.825677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:50.825677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:50.825677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:50.825677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:50.825677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:50.825677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:50.825677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:50.825677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:50.825677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:50.825677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:50.825677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:50.825677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:50.826490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:50.826490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:50.826490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:50.826490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:50.826490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:50.826490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:50.826490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:50.826490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:50.826490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:50.826490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:50.826490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:09:50.826490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:09:50.826490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 110/213] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:50.833668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4d984-41ae-4c47-ad8d-83b02f9a3147 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:50.855314 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:50.926115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d9fb4-18f2-4205-82eb-1b53ccb01389 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:50.953856 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:50.962609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc4a2d-9404-47a3-a3dd-07e76b15c651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:50.963546 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc4a2d-9404-47a3-a3dd-07e76b15c651 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:50.963923 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc4a2d-9404-47a3-a3dd-07e76b15c651 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:50.964298 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc4a2d-9404-47a3-a3dd-07e76b15c651 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:50.965479 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc4a2d-9404-47a3-a3dd-07e76b15c651 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:50.966591 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4d984-41ae-4c47-ad8d-83b02f9a3147 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:50.967070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc4a2d-9404-47a3-a3dd-07e76b15c651 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:50.968981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 104/214] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:50.974119 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc4a2d-9404-47a3-a3dd-07e76b15c651 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:51.011941 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ecc9f-6997-493a-9bbc-724c4bb401bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:51.012577 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ecc9f-6997-493a-9bbc-724c4bb401bf None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:51.012865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ecc9f-6997-493a-9bbc-724c4bb401bf None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:51.013201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ecc9f-6997-493a-9bbc-724c4bb401bf None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:51.013870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ecc9f-6997-493a-9bbc-724c4bb401bf None admin] RBAC: Adding query filter params (name=member) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:51.014170 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ecc9f-6997-493a-9bbc-724c4bb401bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:51.016132 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ecc9f-6997-493a-9bbc-724c4bb401bf None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:51.027261 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 111/215] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:09:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:51.036986 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ecc9f-6997-493a-9bbc-724c4bb401bf None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:51.082290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc4a2d-9404-47a3-a3dd-07e76b15c651 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:51.131535 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:51.133686 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441ad55-528b-4b95-9e9c-496dd4d0744c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:51.134616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441ad55-528b-4b95-9e9c-496dd4d0744c None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:51.135301 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441ad55-528b-4b95-9e9c-496dd4d0744c None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:51.136343 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441ad55-528b-4b95-9e9c-496dd4d0744c None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:51.137210 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441ad55-528b-4b95-9e9c-496dd4d0744c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:51.139455 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441ad55-528b-4b95-9e9c-496dd4d0744c None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:51.148899 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ecc9f-6997-493a-9bbc-724c4bb401bf None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:51.167299 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 105/216] 10.222.0.81 () {60 vars in 1132 bytes} [Mon Sep 19 11:09:50 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:09:51.197819 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b9efd-a66b-442f-80ba-35c1ba23eacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:51.198819 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b9efd-a66b-442f-80ba-35c1ba23eacd None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:51.199276 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b9efd-a66b-442f-80ba-35c1ba23eacd None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:51.199720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b9efd-a66b-442f-80ba-35c1ba23eacd None admin] PATH_INFO: `/v3/users/8db1efb838bc4804be34eb46ec2a001b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:51.216865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b9efd-a66b-442f-80ba-35c1ba23eacd None admin] RBAC: Authorizing `identity:get_user(user_id=8db1efb838bc4804be34eb46ec2a001b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:51.218822 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b9efd-a66b-442f-80ba-35c1ba23eacd None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:51.221254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 112/217] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:51 2022] GET /identity/v3/users/8db1efb838bc4804be34eb46ec2a001b => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:51.227413 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441ad55-528b-4b95-9e9c-496dd4d0744c None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:51.295301 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:51.353185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441ad55-528b-4b95-9e9c-496dd4d0744c None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:51.389265 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed01efa-4b9a-414f-b152-fff2303d7e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:51.389877 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed01efa-4b9a-414f-b152-fff2303d7e72 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:51.390157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed01efa-4b9a-414f-b152-fff2303d7e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:51.390431 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed01efa-4b9a-414f-b152-fff2303d7e72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:51.462746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed01efa-4b9a-414f-b152-fff2303d7e72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:51.464734 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed01efa-4b9a-414f-b152-fff2303d7e72 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:51.522799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:51.575510 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 106/218] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:09:51.581722 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b9efd-a66b-442f-80ba-35c1ba23eacd None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:51.710494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b9efd-a66b-442f-80ba-35c1ba23eacd None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:51.736483 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96771c6-75fc-413e-af53-f13b605a78e4 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:51.736657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96771c6-75fc-413e-af53-f13b605a78e4 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:51.737045 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96771c6-75fc-413e-af53-f13b605a78e4 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:51.738596 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 107/219] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:51.744974 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e403d-31e3-49a4-8548-c6c7108880c6 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:51.745277 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e403d-31e3-49a4-8548-c6c7108880c6 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:51.745522 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e403d-31e3-49a4-8548-c6c7108880c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:51.755671 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d5926-1179-4278-86ce-b6356f8fbe68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:51.758361 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d5926-1179-4278-86ce-b6356f8fbe68 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:51.758361 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d5926-1179-4278-86ce-b6356f8fbe68 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:51.758361 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d5926-1179-4278-86ce-b6356f8fbe68 None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:51.760610 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d5926-1179-4278-86ce-b6356f8fbe68 None admin] RBAC: Authorizing `identity:get_project(project_id=8eb05b28107f4365a8f3ff9b192ce72f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:51.763112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d5926-1179-4278-86ce-b6356f8fbe68 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:51.765718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 113/220] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:51 2022] GET /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:51.768878 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-aa3e403d-31e3-49a4-8548-c6c7108880c6 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:51.770877 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-aa3e403d-31e3-49a4-8548-c6c7108880c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:51.771983 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d5926-1179-4278-86ce-b6356f8fbe68 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:51.792827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:51.849117 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 108/221] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:09:51.890452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19e55b-9bce-4462-a999-578b532867d9 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:51.890666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19e55b-9bce-4462-a999-578b532867d9 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:51.890921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19e55b-9bce-4462-a999-578b532867d9 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:51.892553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 109/222] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:09:51.897238 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cf99c-bb79-4cf2-9a7e-4777f8f03444 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:51.897520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cf99c-bb79-4cf2-9a7e-4777f8f03444 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:51.897792 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cf99c-bb79-4cf2-9a7e-4777f8f03444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:51.900233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d5926-1179-4278-86ce-b6356f8fbe68 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:51.932645 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-253cf99c-bb79-4cf2-9a7e-4777f8f03444 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:51.933841 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-253cf99c-bb79-4cf2-9a7e-4777f8f03444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:51.949357 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064a409-1fd4-4b7e-9e0a-d866ff1f335c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:51.950316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064a409-1fd4-4b7e-9e0a-d866ff1f335c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:51.950799 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064a409-1fd4-4b7e-9e0a-d866ff1f335c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:51.951273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064a409-1fd4-4b7e-9e0a-d866ff1f335c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:51.952231 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064a409-1fd4-4b7e-9e0a-d866ff1f335c None admin] RBAC: Adding query filter params (role.id=0c30c75e227c416da6e4814bce700e8d, scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f, user.id=8db1efb838bc4804be34eb46ec2a001b) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:51.953587 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064a409-1fd4-4b7e-9e0a-d866ff1f335c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:51.956258 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064a409-1fd4-4b7e-9e0a-d866ff1f335c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:51.969976 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 114/223] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:09:51 2022] GET /identity/v3/role_assignments?user.id=8db1efb838bc4804be34eb46ec2a001b&scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f&role.id=0c30c75e227c416da6e4814bce700e8d => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:52.046476 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 110/224] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:09:52.198200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064a409-1fd4-4b7e-9e0a-d866ff1f335c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:52.289505 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:52.330521 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064a409-1fd4-4b7e-9e0a-d866ff1f335c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:52.359207 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cf99c-bb79-4cf2-9a7e-4777f8f03444 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:52.375335 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:52.387180 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25879b-57c2-4c01-8270-f33722e7a098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:52.387848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25879b-57c2-4c01-8270-f33722e7a098 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:52.388129 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25879b-57c2-4c01-8270-f33722e7a098 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:52.388402 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25879b-57c2-4c01-8270-f33722e7a098 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:52.408049 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25879b-57c2-4c01-8270-f33722e7a098 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:52.413435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-cb25879b-57c2-4c01-8270-f33722e7a098 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:52.413435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:52.413435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:52.413435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:52.413435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:52.413435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:52.413435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:52.413435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:52.413435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:52.413435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:52.413435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:52.413435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:52.413435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:52.413435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:52.413435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:52.413435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:52.414324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:52.414324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:52.414324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:52.414324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:52.414324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:52.414324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:52.414324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:52.414324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:52.414324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:52.414324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:52.414324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:52.414324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:52.414324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:52.414324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:52.414324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:52.414324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:52.415117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:52.415117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:52.415117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:52.415117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:52.415117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:52.415117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:52.415117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:52.415117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:52.415117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:52.415117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:52.415117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:52.415117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:09:52.420478 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 115/225] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:09:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:52.421344 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25879b-57c2-4c01-8270-f33722e7a098 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:52.525899 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cf99c-bb79-4cf2-9a7e-4777f8f03444 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:52.597395 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eaf454-f916-483f-9914-0bc2b33fce8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:52.597944 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaf454-f916-483f-9914-0bc2b33fce8e None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:52.598220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaf454-f916-483f-9914-0bc2b33fce8e None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:52.598488 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaf454-f916-483f-9914-0bc2b33fce8e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:52.608305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25879b-57c2-4c01-8270-f33722e7a098 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:52.624097 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eaf454-f916-483f-9914-0bc2b33fce8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:52.629562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-91eaf454-f916-483f-9914-0bc2b33fce8e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:09:52.629562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:52.629562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:52.629562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:52.629562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:52.629562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:52.629562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:52.629562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:52.629562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:52.629562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:52.629562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:52.629562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:52.629562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:52.629562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:52.629562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:52.629562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:52.630563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:52.630563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:52.630563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:52.630563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:52.630563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:52.630563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:52.630563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:52.630563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:52.630563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:52.630563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:52.630563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:52.630563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:52.630563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:52.630563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:52.630563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:52.630563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:52.631402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:52.631402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:52.631402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:52.631402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:52.631402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:52.631402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:52.631402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:52.631402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:52.631402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:52.631402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:52.631402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:09:52.631402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:09:52.631402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 111/226] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:09:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:09:52.640363 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eaf454-f916-483f-9914-0bc2b33fce8e None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:52.667868 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:52.682952 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8aac35-b8f2-411b-af00-13c0e22cf0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:52.683568 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8aac35-b8f2-411b-af00-13c0e22cf0a2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:52.683868 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8aac35-b8f2-411b-af00-13c0e22cf0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:52.684140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8aac35-b8f2-411b-af00-13c0e22cf0a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:52.684808 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8aac35-b8f2-411b-af00-13c0e22cf0a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:52.685112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8aac35-b8f2-411b-af00-13c0e22cf0a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:52.686504 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8aac35-b8f2-411b-af00-13c0e22cf0a2 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:52.706869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 116/227] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:52.712017 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8aac35-b8f2-411b-af00-13c0e22cf0a2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:52.784110 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:52.829789 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eaf454-f916-483f-9914-0bc2b33fce8e None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:52.896683 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f382b842-acd4-406a-a73a-32a601adf41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:52.897281 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382b842-acd4-406a-a73a-32a601adf41f None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:52.897557 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382b842-acd4-406a-a73a-32a601adf41f None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:52.897825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382b842-acd4-406a-a73a-32a601adf41f None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:52.898484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f382b842-acd4-406a-a73a-32a601adf41f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:52.898792 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f382b842-acd4-406a-a73a-32a601adf41f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:52.900197 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f382b842-acd4-406a-a73a-32a601adf41f None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:52.904095 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8aac35-b8f2-411b-af00-13c0e22cf0a2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:52.918427 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 112/228] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:09:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:52.923904 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f382b842-acd4-406a-a73a-32a601adf41f None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:52.968311 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df7840-4754-4095-95c2-60310ec535fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:52.968947 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df7840-4754-4095-95c2-60310ec535fe None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:52.969221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df7840-4754-4095-95c2-60310ec535fe None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:52.969489 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df7840-4754-4095-95c2-60310ec535fe None admin] PATH_INFO: `/v3/users/aeaea0c2d8494dfab06bfd3933c13ab0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:52.970735 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df7840-4754-4095-95c2-60310ec535fe None admin] RBAC: Authorizing `identity:get_user(user_id=aeaea0c2d8494dfab06bfd3933c13ab0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:52.972221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df7840-4754-4095-95c2-60310ec535fe None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:52.974475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 117/229] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:52 2022] GET /identity/v3/users/aeaea0c2d8494dfab06bfd3933c13ab0 => generated 283 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:52.984692 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df7840-4754-4095-95c2-60310ec535fe None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:53.085310 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:53.121723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f382b842-acd4-406a-a73a-32a601adf41f None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:53.150421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df7840-4754-4095-95c2-60310ec535fe None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:53.194274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35948649-9534-41da-a6ca-4ace56515d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:53.194908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35948649-9534-41da-a6ca-4ace56515d55 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:53.195187 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35948649-9534-41da-a6ca-4ace56515d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:53.195452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35948649-9534-41da-a6ca-4ace56515d55 None admin] PATH_INFO: `/v3/users/aeaea0c2d8494dfab06bfd3933c13ab0` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:53.196606 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35948649-9534-41da-a6ca-4ace56515d55 None admin] RBAC: Authorizing `identity:get_user(user_id=aeaea0c2d8494dfab06bfd3933c13ab0)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:53.198565 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35948649-9534-41da-a6ca-4ace56515d55 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:53.203815 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 113/230] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:52 2022] GET /identity/v3/users/aeaea0c2d8494dfab06bfd3933c13ab0 => generated 283 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:53.206750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35948649-9534-41da-a6ca-4ace56515d55 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:53.209191 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331fb28-e626-4be4-b8a9-0c8c5d8dfc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:53.209786 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:53.210895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331fb28-e626-4be4-b8a9-0c8c5d8dfc66 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:53.211377 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331fb28-e626-4be4-b8a9-0c8c5d8dfc66 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:53.211851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331fb28-e626-4be4-b8a9-0c8c5d8dfc66 None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:53.214102 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331fb28-e626-4be4-b8a9-0c8c5d8dfc66 None admin] RBAC: Authorizing `identity:get_project(project_id=8eb05b28107f4365a8f3ff9b192ce72f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:53.215874 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331fb28-e626-4be4-b8a9-0c8c5d8dfc66 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:53.218043 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 118/231] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:52 2022] GET /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:53.225919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331fb28-e626-4be4-b8a9-0c8c5d8dfc66 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:53.373240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331fb28-e626-4be4-b8a9-0c8c5d8dfc66 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:53.394243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35948649-9534-41da-a6ca-4ace56515d55 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:53.439717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f333b-20ba-426f-a8c5-9fcb61e41f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:53.440686 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f333b-20ba-426f-a8c5-9fcb61e41f56 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:53.441147 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f333b-20ba-426f-a8c5-9fcb61e41f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:53.441595 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f333b-20ba-426f-a8c5-9fcb61e41f56 None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:53.443661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f333b-20ba-426f-a8c5-9fcb61e41f56 None admin] RBAC: Authorizing `identity:get_project(project_id=8eb05b28107f4365a8f3ff9b192ce72f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:53.445373 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f333b-20ba-426f-a8c5-9fcb61e41f56 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:53.447378 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:53.450045 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 114/232] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:53 2022] GET /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:53.460879 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f333b-20ba-426f-a8c5-9fcb61e41f56 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:53.474008 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85049f1-3bb6-4623-a240-71390eeb7ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:53.475015 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85049f1-3bb6-4623-a240-71390eeb7ca8 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:53.475500 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85049f1-3bb6-4623-a240-71390eeb7ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:53.475984 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85049f1-3bb6-4623-a240-71390eeb7ca8 None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f/users/aeaea0c2d8494dfab06bfd3933c13ab0/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:53.480374 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85049f1-3bb6-4623-a240-71390eeb7ca8 None admin] RBAC: Authorizing `identity:create_grant(project_id=8eb05b28107f4365a8f3ff9b192ce72f, user_id=aeaea0c2d8494dfab06bfd3933c13ab0, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:53.482034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85049f1-3bb6-4623-a240-71390eeb7ca8 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:53.510820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 119/233] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:09:53 2022] PUT /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f/users/aeaea0c2d8494dfab06bfd3933c13ab0/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:09:53.564548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:53.605402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f333b-20ba-426f-a8c5-9fcb61e41f56 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:53.652093 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27899201-16db-46d2-ad2e-5c0dd0ad39a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:53.653150 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27899201-16db-46d2-ad2e-5c0dd0ad39a0 None admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:53.653634 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27899201-16db-46d2-ad2e-5c0dd0ad39a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:53.654142 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27899201-16db-46d2-ad2e-5c0dd0ad39a0 None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f/users/aeaea0c2d8494dfab06bfd3933c13ab0/roles/4cc230d31ac94f4c89bf827491c01bd7` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:53.657511 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27899201-16db-46d2-ad2e-5c0dd0ad39a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=8eb05b28107f4365a8f3ff9b192ce72f, user_id=aeaea0c2d8494dfab06bfd3933c13ab0, role_id=4cc230d31ac94f4c89bf827491c01bd7)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:53.659289 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27899201-16db-46d2-ad2e-5c0dd0ad39a0 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:53.699109 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 115/234] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:09:53 2022] PUT /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f/users/aeaea0c2d8494dfab06bfd3933c13ab0/roles/4cc230d31ac94f4c89bf827491c01bd7 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:09:54.047561 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26c72e-c057-4a93-8728-a5f510bfdd8f None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:54.048095 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328d28f-feb3-47f6-b342-1e1b99546b2b None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:54.048174 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328d28f-feb3-47f6-b342-1e1b99546b2b None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:54.048437 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328d28f-feb3-47f6-b342-1e1b99546b2b None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:54.048931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26c72e-c057-4a93-8728-a5f510bfdd8f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:54.049430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26c72e-c057-4a93-8728-a5f510bfdd8f None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:54.050039 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 116/235] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:54.051190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 120/236] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:54.055327 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411c5f6-0779-4364-ad8e-1f3f1eca4644 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:54.055616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411c5f6-0779-4364-ad8e-1f3f1eca4644 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:54.055927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411c5f6-0779-4364-ad8e-1f3f1eca4644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:54.057304 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f0212-8895-4837-92bf-babe03616bad None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:54.057593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f0212-8895-4837-92bf-babe03616bad None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:54.057868 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f0212-8895-4837-92bf-babe03616bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:54.078890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-9411c5f6-0779-4364-ad8e-1f3f1eca4644 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:54.080117 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-9411c5f6-0779-4364-ad8e-1f3f1eca4644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:54.083893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-119f0212-8895-4837-92bf-babe03616bad None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:54.085156 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-119f0212-8895-4837-92bf-babe03616bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:54.185675 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 117/237] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:54.188334 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 121/238] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:54.455135 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411c5f6-0779-4364-ad8e-1f3f1eca4644 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:54.530060 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:54.563837 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59060cc7-d48f-4f92-9b25-e742ca163200 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:54.564154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59060cc7-d48f-4f92-9b25-e742ca163200 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:54.564386 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59060cc7-d48f-4f92-9b25-e742ca163200 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:54.565723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 122/239] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:09:54.570688 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223ad4a-b549-4dc9-86e8-f951ca9a6db9 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:54.570969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223ad4a-b549-4dc9-86e8-f951ca9a6db9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:54.571245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223ad4a-b549-4dc9-86e8-f951ca9a6db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:54.587029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411c5f6-0779-4364-ad8e-1f3f1eca4644 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:54.589964 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-3223ad4a-b549-4dc9-86e8-f951ca9a6db9 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:54.591148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3223ad4a-b549-4dc9-86e8-f951ca9a6db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:54.656942 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c21a5-df32-4e68-9a02-6cb8678738c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:54.657597 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c21a5-df32-4e68-9a02-6cb8678738c1 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:54.657861 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c21a5-df32-4e68-9a02-6cb8678738c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:54.658130 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c21a5-df32-4e68-9a02-6cb8678738c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:54.659273 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c21a5-df32-4e68-9a02-6cb8678738c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:54.669612 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c21a5-df32-4e68-9a02-6cb8678738c1 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:54.671886 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 118/240] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:54.676568 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 123/241] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:09:54.677511 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c21a5-df32-4e68-9a02-6cb8678738c1 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:54.753803 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223ad4a-b549-4dc9-86e8-f951ca9a6db9 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:54.787895 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c21a5-df32-4e68-9a02-6cb8678738c1 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:54.816200 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:54.825032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788904f4-ef9e-4c91-8e65-bdb9ce1e63dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:54.825989 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788904f4-ef9e-4c91-8e65-bdb9ce1e63dd None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:54.826461 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788904f4-ef9e-4c91-8e65-bdb9ce1e63dd None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:54.827019 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788904f4-ef9e-4c91-8e65-bdb9ce1e63dd None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:54.827899 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788904f4-ef9e-4c91-8e65-bdb9ce1e63dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:54.829778 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788904f4-ef9e-4c91-8e65-bdb9ce1e63dd None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:54.851319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:54.875504 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 119/242] 10.222.0.81 () {60 vars in 1132 bytes} [Mon Sep 19 11:09:54 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:09:54.923530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223ad4a-b549-4dc9-86e8-f951ca9a6db9 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:54.968810 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a994fd-bf5f-4fc2-bb7c-5913d2a3b600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:54.969806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a994fd-bf5f-4fc2-bb7c-5913d2a3b600 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:54.970296 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a994fd-bf5f-4fc2-bb7c-5913d2a3b600 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:54.970780 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a994fd-bf5f-4fc2-bb7c-5913d2a3b600 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:54.981527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a994fd-bf5f-4fc2-bb7c-5913d2a3b600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:54.987052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-b8a994fd-bf5f-4fc2-bb7c-5913d2a3b600 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:09:54.987052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:54.987052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:54.987052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:54.987052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:54.987052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:54.987052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:54.987052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:54.987052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:54.987052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:54.987052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:54.987052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:54.987052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:54.987052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:54.987052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:54.987052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:54.988012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:54.988012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:54.988012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:54.988012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:54.988012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:54.988012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:54.988012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:54.988012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:54.988012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:54.988012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:54.988012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:54.988012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:54.988012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:54.988012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:54.988012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:54.988012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:54.988839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:54.988839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:54.988839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:54.988839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:54.988839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:54.988839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:54.988839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:54.988839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:54.988839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:54.988839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:54.988839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:09:54.988839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:09:54.991600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 124/243] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:09:55.001048 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788904f4-ef9e-4c91-8e65-bdb9ce1e63dd None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:55.058586 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a994fd-bf5f-4fc2-bb7c-5913d2a3b600 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:55.132809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788904f4-ef9e-4c91-8e65-bdb9ce1e63dd None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:55.169503 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f051953-a90e-4b59-9b8c-dd4b415d60ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:55.170274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f051953-a90e-4b59-9b8c-dd4b415d60ec None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:55.170677 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f051953-a90e-4b59-9b8c-dd4b415d60ec None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:55.171053 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f051953-a90e-4b59-9b8c-dd4b415d60ec None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:55.171860 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f051953-a90e-4b59-9b8c-dd4b415d60ec None admin] RBAC: Adding query filter params (name=reader) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:55.172265 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f051953-a90e-4b59-9b8c-dd4b415d60ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:55.173736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f051953-a90e-4b59-9b8c-dd4b415d60ec None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:55.182503 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 120/244] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:09:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:55.189149 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f051953-a90e-4b59-9b8c-dd4b415d60ec None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:55.209717 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a994fd-bf5f-4fc2-bb7c-5913d2a3b600 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:55.227048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:55.248161 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:55.257805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93bfb7-9f73-4711-9cba-059677b0982b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:55.258804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93bfb7-9f73-4711-9cba-059677b0982b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:55.259715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93bfb7-9f73-4711-9cba-059677b0982b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:55.260616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93bfb7-9f73-4711-9cba-059677b0982b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:55.275143 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93bfb7-9f73-4711-9cba-059677b0982b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:55.281866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-2d93bfb7-9f73-4711-9cba-059677b0982b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:09:55.281866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:55.281866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:55.281866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:55.281866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:55.281866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:55.281866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:55.281866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:55.281866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:55.281866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:55.281866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:55.281866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:55.281866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:55.281866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:55.281866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:55.281866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:55.282769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:55.282769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:55.282769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:55.282769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:55.282769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:55.282769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:55.282769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:55.282769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:55.282769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:55.282769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:55.282769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:55.282769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:55.282769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:55.282769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:55.282769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:55.282769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:55.283699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:55.283699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:55.283699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:55.283699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:55.283699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:55.283699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:55.283699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:55.283699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:55.283699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:55.283699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:55.283699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:09:55.283699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:09:55.286211 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 125/245] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:09:55.292391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93bfb7-9f73-4711-9cba-059677b0982b None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:55.358577 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f051953-a90e-4b59-9b8c-dd4b415d60ec None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:55.404109 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d35b3-4382-404d-88a3-459b8ed1a3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:55.404999 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d35b3-4382-404d-88a3-459b8ed1a3b1 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:55.405419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d35b3-4382-404d-88a3-459b8ed1a3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:55.406271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d35b3-4382-404d-88a3-459b8ed1a3b1 None admin] PATH_INFO: `/v3/users/3dc65c481dd34193b57064c161c65f81` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:55.428210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93bfb7-9f73-4711-9cba-059677b0982b None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:55.428521 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d35b3-4382-404d-88a3-459b8ed1a3b1 None admin] RBAC: Authorizing `identity:get_user(user_id=3dc65c481dd34193b57064c161c65f81)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:55.430299 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d35b3-4382-404d-88a3-459b8ed1a3b1 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:55.433836 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 121/246] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:55 2022] GET /identity/v3/users/3dc65c481dd34193b57064c161c65f81 => generated 297 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:55.440332 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d35b3-4382-404d-88a3-459b8ed1a3b1 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:55.468400 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f87d8-5848-4d4a-9a85-e9ded200a67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:55.469185 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f87d8-5848-4d4a-9a85-e9ded200a67c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:55.469761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f87d8-5848-4d4a-9a85-e9ded200a67c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:55.470042 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f87d8-5848-4d4a-9a85-e9ded200a67c None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:55.470989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125f87d8-5848-4d4a-9a85-e9ded200a67c None admin] RBAC: Adding query filter params (name=member) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:55.471294 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125f87d8-5848-4d4a-9a85-e9ded200a67c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:55.473570 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125f87d8-5848-4d4a-9a85-e9ded200a67c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:55.488104 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 126/247] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:09:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:55.490995 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f87d8-5848-4d4a-9a85-e9ded200a67c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:55.532408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:55.562266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d35b3-4382-404d-88a3-459b8ed1a3b1 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:55.570092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:55.600421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda84ae-5ba8-4abd-94c4-a586081dce99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:55.601175 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda84ae-5ba8-4abd-94c4-a586081dce99 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:55.601558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda84ae-5ba8-4abd-94c4-a586081dce99 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:55.601935 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda84ae-5ba8-4abd-94c4-a586081dce99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:55.644277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f87d8-5848-4d4a-9a85-e9ded200a67c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:55.712472 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1d1e4-78ca-4167-93cc-530c0564388a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:55.714424 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1d1e4-78ca-4167-93cc-530c0564388a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:55.717218 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1d1e4-78ca-4167-93cc-530c0564388a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:55.717695 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1d1e4-78ca-4167-93cc-530c0564388a None admin] PATH_INFO: `/v3/users/8db1efb838bc4804be34eb46ec2a001b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:55.718998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1d1e4-78ca-4167-93cc-530c0564388a None admin] RBAC: Authorizing `identity:get_user(user_id=8db1efb838bc4804be34eb46ec2a001b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:55.720644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1d1e4-78ca-4167-93cc-530c0564388a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:55.725194 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 127/248] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:55 2022] GET /identity/v3/users/8db1efb838bc4804be34eb46ec2a001b => generated 297 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:55.726345 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda84ae-5ba8-4abd-94c4-a586081dce99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:55.727956 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda84ae-5ba8-4abd-94c4-a586081dce99 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:55.730742 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1d1e4-78ca-4167-93cc-530c0564388a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:55.884759 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1d1e4-78ca-4167-93cc-530c0564388a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:55.912130 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:55.951366 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c247b16-3cae-47fd-b420-eee7c9ab754c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:55.952080 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c247b16-3cae-47fd-b420-eee7c9ab754c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:55.952361 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c247b16-3cae-47fd-b420-eee7c9ab754c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:55.952634 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c247b16-3cae-47fd-b420-eee7c9ab754c None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:55.954091 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c247b16-3cae-47fd-b420-eee7c9ab754c None admin] RBAC: Authorizing `identity:get_project(project_id=8eb05b28107f4365a8f3ff9b192ce72f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:55.955562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c247b16-3cae-47fd-b420-eee7c9ab754c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:55.957324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 128/249] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:55 2022] GET /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:55.969607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c247b16-3cae-47fd-b420-eee7c9ab754c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:55.977308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 122/250] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 11:09:55.989068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda84ae-5ba8-4abd-94c4-a586081dce99 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:55.992175 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:56.140735 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c247b16-3cae-47fd-b420-eee7c9ab754c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:56.175906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda84ae-5ba8-4abd-94c4-a586081dce99 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:56.196929 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e031e7-a175-411c-b9e8-377f275d22fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:56.197652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e031e7-a175-411c-b9e8-377f275d22fe None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:56.197927 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e031e7-a175-411c-b9e8-377f275d22fe None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:56.198196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e031e7-a175-411c-b9e8-377f275d22fe None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f/users/8db1efb838bc4804be34eb46ec2a001b/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:56.200142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e031e7-a175-411c-b9e8-377f275d22fe None admin] RBAC: Authorizing `identity:create_grant(project_id=8eb05b28107f4365a8f3ff9b192ce72f, user_id=8db1efb838bc4804be34eb46ec2a001b, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:56.201594 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e031e7-a175-411c-b9e8-377f275d22fe None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:56.220849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 129/251] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:09:55 2022] PUT /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f/users/8db1efb838bc4804be34eb46ec2a001b/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:09:56.248619 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ef0f3-80c6-4b80-b5fa-a0b306f17aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:56.249295 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ef0f3-80c6-4b80-b5fa-a0b306f17aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:56.249579 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ef0f3-80c6-4b80-b5fa-a0b306f17aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:56.249857 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ef0f3-80c6-4b80-b5fa-a0b306f17aa3 None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:56.251283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ef0f3-80c6-4b80-b5fa-a0b306f17aa3 None admin] RBAC: Authorizing `identity:get_project(project_id=8eb05b28107f4365a8f3ff9b192ce72f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:56.252912 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ef0f3-80c6-4b80-b5fa-a0b306f17aa3 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:56.254994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 123/252] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:55 2022] GET /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:56.261321 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e031e7-a175-411c-b9e8-377f275d22fe None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:56.305068 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:56.375860 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5c7e6-114c-4411-b36e-f01e294f5be1 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:56.376200 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5c7e6-114c-4411-b36e-f01e294f5be1 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:56.376415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5c7e6-114c-4411-b36e-f01e294f5be1 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:56.378063 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 124/253] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:56.393573 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c8aed-d4dc-4f69-a8dc-e9ce9897729b None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:56.393852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c8aed-d4dc-4f69-a8dc-e9ce9897729b None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:56.394133 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c8aed-d4dc-4f69-a8dc-e9ce9897729b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:56.415412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e031e7-a175-411c-b9e8-377f275d22fe None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:56.440473 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-1e5c8aed-d4dc-4f69-a8dc-e9ce9897729b None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:56.441613 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-1e5c8aed-d4dc-4f69-a8dc-e9ce9897729b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:56.493480 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1376bb-5c1e-484a-8eb5-11ffd1e9a502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:56.494174 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1376bb-5c1e-484a-8eb5-11ffd1e9a502 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:56.495007 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1376bb-5c1e-484a-8eb5-11ffd1e9a502 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:56.495283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1376bb-5c1e-484a-8eb5-11ffd1e9a502 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:56.496054 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1376bb-5c1e-484a-8eb5-11ffd1e9a502 None admin] RBAC: Adding query filter params (role.id=967fbd121d5e4d65b135451eaf2c03d8, scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f, user.id=3dc65c481dd34193b57064c161c65f81) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:56.496370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1376bb-5c1e-484a-8eb5-11ffd1e9a502 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:56.498107 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1376bb-5c1e-484a-8eb5-11ffd1e9a502 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:56.509592 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 130/254] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:09:56 2022] GET /identity/v3/role_assignments?user.id=3dc65c481dd34193b57064c161c65f81&scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f&role.id=967fbd121d5e4d65b135451eaf2c03d8 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:56.540530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd6bda-189d-494a-949c-db3b659ef0b3 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:56.540838 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd6bda-189d-494a-949c-db3b659ef0b3 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:56.541105 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd6bda-189d-494a-949c-db3b659ef0b3 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:56.542480 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 131/255] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:09:56.548810 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf648b-31c2-4282-adcd-5701fe21ff67 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:56.549108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf648b-31c2-4282-adcd-5701fe21ff67 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:56.549548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf648b-31c2-4282-adcd-5701fe21ff67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:56.575325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-50cf648b-31c2-4282-adcd-5701fe21ff67 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:56.576796 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-50cf648b-31c2-4282-adcd-5701fe21ff67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:56.588258 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 125/256] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:56.667098 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 132/257] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:09:56.985543 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c8aed-d4dc-4f69-a8dc-e9ce9897729b None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:56.988316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:57.122169 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c8aed-d4dc-4f69-a8dc-e9ce9897729b None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:57.168469 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8497d78-f0b2-4518-ac24-e9e325bdc120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:57.169106 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8497d78-f0b2-4518-ac24-e9e325bdc120 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:57.169379 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8497d78-f0b2-4518-ac24-e9e325bdc120 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:57.169645 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8497d78-f0b2-4518-ac24-e9e325bdc120 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:57.180557 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8497d78-f0b2-4518-ac24-e9e325bdc120 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:57.182636 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf648b-31c2-4282-adcd-5701fe21ff67 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:57.186991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-a8497d78-f0b2-4518-ac24-e9e325bdc120 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:09:57.186991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:57.186991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:57.186991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:57.186991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:57.186991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:57.186991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:57.186991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:57.186991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:57.186991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:57.186991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:57.186991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:57.186991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:57.186991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:57.186991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:57.186991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:57.187920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:57.187920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:57.187920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:57.187920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:57.187920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:57.187920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:57.187920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:57.187920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:57.187920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:57.187920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:57.187920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:57.187920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:57.187920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:57.187920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:57.187920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:57.187920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:57.188715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:57.188715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:57.188715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:57.188715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:57.188715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:57.188715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:57.188715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:57.188715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:57.188715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:57.188715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:57.188715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:09:57.188715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:09:57.188715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 126/258] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:09:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:09:57.195483 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8497d78-f0b2-4518-ac24-e9e325bdc120 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:57.237293 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:57.317548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:57.410541 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8497d78-f0b2-4518-ac24-e9e325bdc120 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:57.427622 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf648b-31c2-4282-adcd-5701fe21ff67 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:57.479175 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08624d7c-7c23-4e60-80e3-44a944d3689f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:57.479819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08624d7c-7c23-4e60-80e3-44a944d3689f None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:57.480108 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08624d7c-7c23-4e60-80e3-44a944d3689f None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:57.480375 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08624d7c-7c23-4e60-80e3-44a944d3689f None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:57.481041 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08624d7c-7c23-4e60-80e3-44a944d3689f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:57.481338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08624d7c-7c23-4e60-80e3-44a944d3689f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:57.482709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08624d7c-7c23-4e60-80e3-44a944d3689f None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:57.491589 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 127/259] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:09:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:09:57.496729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08624d7c-7c23-4e60-80e3-44a944d3689f None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:57.501462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f1d0b-2bbf-4fb8-a626-06c727422cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:57.502121 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f1d0b-2bbf-4fb8-a626-06c727422cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:57.502398 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f1d0b-2bbf-4fb8-a626-06c727422cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:57.502671 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f1d0b-2bbf-4fb8-a626-06c727422cf6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:57.532680 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f1d0b-2bbf-4fb8-a626-06c727422cf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:57.537649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-f12f1d0b-2bbf-4fb8-a626-06c727422cf6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:57.537649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:57.537649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:57.537649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:57.537649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:57.537649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:57.537649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:57.537649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:57.537649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:57.537649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:57.537649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:57.537649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:57.537649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:57.537649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:57.537649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:57.537649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:57.538616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:57.538616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:57.538616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:57.538616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:57.538616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:57.538616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:57.538616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:57.538616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:57.538616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:57.538616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:57.538616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:57.538616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:57.538616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:57.538616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:57.538616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:57.538616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:57.539417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:57.539417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:57.539417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:57.539417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:57.539417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:57.539417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:57.539417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:57.539417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:57.539417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:57.539417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:57.539417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:09:57.539417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:09:57.539417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 133/260] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:09:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:57.548697 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f1d0b-2bbf-4fb8-a626-06c727422cf6 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:57.649208 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08624d7c-7c23-4e60-80e3-44a944d3689f None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:57.699147 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:57.723803 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07a802-99cb-4fc8-b19c-1174402084eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:57.724449 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07a802-99cb-4fc8-b19c-1174402084eb None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:57.724729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07a802-99cb-4fc8-b19c-1174402084eb None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:57.725106 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07a802-99cb-4fc8-b19c-1174402084eb None admin] PATH_INFO: `/v3/users/aeaea0c2d8494dfab06bfd3933c13ab0` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:57.726249 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07a802-99cb-4fc8-b19c-1174402084eb None admin] RBAC: Authorizing `identity:get_user(user_id=aeaea0c2d8494dfab06bfd3933c13ab0)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:57.728076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07a802-99cb-4fc8-b19c-1174402084eb None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:57.736070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 128/261] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:57 2022] GET /identity/v3/users/aeaea0c2d8494dfab06bfd3933c13ab0 => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:57.741266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07a802-99cb-4fc8-b19c-1174402084eb None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:57.763233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:57.810144 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f1d0b-2bbf-4fb8-a626-06c727422cf6 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:57.884572 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac41479-eba8-4d20-8711-1ba8f0465ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:57.885789 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac41479-eba8-4d20-8711-1ba8f0465ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:57.886055 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac41479-eba8-4d20-8711-1ba8f0465ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:57.886323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac41479-eba8-4d20-8711-1ba8f0465ba2 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:57.887010 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac41479-eba8-4d20-8711-1ba8f0465ba2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:57.887306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac41479-eba8-4d20-8711-1ba8f0465ba2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:57.888767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac41479-eba8-4d20-8711-1ba8f0465ba2 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:57.904242 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 134/262] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:57.909584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac41479-eba8-4d20-8711-1ba8f0465ba2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:57.977923 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07a802-99cb-4fc8-b19c-1174402084eb None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:58.086432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74caa9-989d-459e-837a-0a3fa20100b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:58.086968 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74caa9-989d-459e-837a-0a3fa20100b4 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:58.087248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74caa9-989d-459e-837a-0a3fa20100b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:58.087519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74caa9-989d-459e-837a-0a3fa20100b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:58.119689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac41479-eba8-4d20-8711-1ba8f0465ba2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:58.205035 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4a9a4-5d8c-4b75-82c1-8fa8248b07a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:58.205670 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4a9a4-5d8c-4b75-82c1-8fa8248b07a9 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:58.205937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4a9a4-5d8c-4b75-82c1-8fa8248b07a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:58.206231 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4a9a4-5d8c-4b75-82c1-8fa8248b07a9 None admin] PATH_INFO: `/v3/users/aeaea0c2d8494dfab06bfd3933c13ab0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:58.212488 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4a9a4-5d8c-4b75-82c1-8fa8248b07a9 None admin] RBAC: Authorizing `identity:get_user(user_id=aeaea0c2d8494dfab06bfd3933c13ab0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:58.214505 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4a9a4-5d8c-4b75-82c1-8fa8248b07a9 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:58.216442 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 135/263] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:57 2022] GET /identity/v3/users/aeaea0c2d8494dfab06bfd3933c13ab0 => generated 283 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:58.224926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4a9a4-5d8c-4b75-82c1-8fa8248b07a9 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:58.226575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:58.239029 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:58.249046 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74caa9-989d-459e-837a-0a3fa20100b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:58.250923 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74caa9-989d-459e-837a-0a3fa20100b4 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:58.376003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 129/264] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:09:58.386446 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74caa9-989d-459e-837a-0a3fa20100b4 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:58.409153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4a9a4-5d8c-4b75-82c1-8fa8248b07a9 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:58.488531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77228ff-b859-4418-876e-1630d7eb7690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:58.489160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77228ff-b859-4418-876e-1630d7eb7690 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:58.489436 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77228ff-b859-4418-876e-1630d7eb7690 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:58.489714 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77228ff-b859-4418-876e-1630d7eb7690 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:58.499120 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:58.521913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74caa9-989d-459e-837a-0a3fa20100b4 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:58.581682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374a33b-b2c6-4f2b-bb25-8d7050949ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:58.582334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374a33b-b2c6-4f2b-bb25-8d7050949ead None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:58.582612 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374a33b-b2c6-4f2b-bb25-8d7050949ead None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:58.582881 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374a33b-b2c6-4f2b-bb25-8d7050949ead None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:58.585604 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374a33b-b2c6-4f2b-bb25-8d7050949ead None admin] RBAC: Authorizing `identity:get_project(project_id=8eb05b28107f4365a8f3ff9b192ce72f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:58.587077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374a33b-b2c6-4f2b-bb25-8d7050949ead None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:58.589438 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 130/265] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:58 2022] GET /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:58.596179 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374a33b-b2c6-4f2b-bb25-8d7050949ead None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:58.647626 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77228ff-b859-4418-876e-1630d7eb7690 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:58.649473 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77228ff-b859-4418-876e-1630d7eb7690 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:58.665138 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:58.774415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374a33b-b2c6-4f2b-bb25-8d7050949ead None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:58.786633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 136/266] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:09:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 11:09:58.787724 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6230e-e952-42e5-a00a-cab0479a4e17 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:58.788036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6230e-e952-42e5-a00a-cab0479a4e17 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:58.788305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6230e-e952-42e5-a00a-cab0479a4e17 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:58.790127 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 137/267] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:58.797408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fb618-e7ba-4a19-b46a-18739edc189f None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:58.797408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fb618-e7ba-4a19-b46a-18739edc189f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:58.797408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fb618-e7ba-4a19-b46a-18739edc189f None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:58.797408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 138/268] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:58.797408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820fb618-e7ba-4a19-b46a-18739edc189f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:58.839881 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9c97d-84e1-440f-826f-a058463c88bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:58.840841 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9c97d-84e1-440f-826f-a058463c88bd None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:58.841349 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9c97d-84e1-440f-826f-a058463c88bd None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:58.841833 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9c97d-84e1-440f-826f-a058463c88bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:58.843356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9c97d-84e1-440f-826f-a058463c88bd None admin] RBAC: Adding query filter params (role.id=4cc230d31ac94f4c89bf827491c01bd7, scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f, user.id=aeaea0c2d8494dfab06bfd3933c13ab0) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:58.843903 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9c97d-84e1-440f-826f-a058463c88bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:58.845545 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9c97d-84e1-440f-826f-a058463c88bd None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:58.856598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 131/269] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:09:58 2022] GET /identity/v3/role_assignments?user.id=aeaea0c2d8494dfab06bfd3933c13ab0&scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f&role.id=4cc230d31ac94f4c89bf827491c01bd7 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:58.858270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5898ffd-9446-4d8a-aec0-3cfa110e3ea6 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:58.858757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5898ffd-9446-4d8a-aec0-3cfa110e3ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:58.859231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5898ffd-9446-4d8a-aec0-3cfa110e3ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:58.900186 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-c5898ffd-9446-4d8a-aec0-3cfa110e3ea6 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:58.902274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c5898ffd-9446-4d8a-aec0-3cfa110e3ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:58.935644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820fb618-e7ba-4a19-b46a-18739edc189f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:58.952139 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:58.963791 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:59.022178 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755505f-56ce-458e-87d3-5f389cebe34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:59.022982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755505f-56ce-458e-87d3-5f389cebe34b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:59.024768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755505f-56ce-458e-87d3-5f389cebe34b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:59.024768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755505f-56ce-458e-87d3-5f389cebe34b None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:59.028207 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755505f-56ce-458e-87d3-5f389cebe34b None admin] RBAC: Authorizing `identity:get_project(project_id=8eb05b28107f4365a8f3ff9b192ce72f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:59.029773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755505f-56ce-458e-87d3-5f389cebe34b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:59.037142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 139/270] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:09:58 2022] GET /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:59.038543 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3c673-3267-44f7-b710-c215bb325e6c None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:59.038823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3c673-3267-44f7-b710-c215bb325e6c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:59.039211 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3c673-3267-44f7-b710-c215bb325e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:59.074011 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 132/271] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:59.074708 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5898ffd-9446-4d8a-aec0-3cfa110e3ea6 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:59.086052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-7dc3c673-3267-44f7-b710-c215bb325e6c None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:59.090965 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7dc3c673-3267-44f7-b710-c215bb325e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:59.219224 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 140/272] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:09:59.242064 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5898ffd-9446-4d8a-aec0-3cfa110e3ea6 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:59.293846 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b156fc-4565-4d7f-86be-1b99d1164545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:59.294895 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b156fc-4565-4d7f-86be-1b99d1164545 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:59.295378 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b156fc-4565-4d7f-86be-1b99d1164545 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:59.295918 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b156fc-4565-4d7f-86be-1b99d1164545 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:59.296884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b156fc-4565-4d7f-86be-1b99d1164545 None admin] RBAC: Adding query filter params (role.id=753f8b35d5f0458d8f62d0ad6e28033f, scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f, user.id=aeaea0c2d8494dfab06bfd3933c13ab0) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:59.297438 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b156fc-4565-4d7f-86be-1b99d1164545 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:59.299164 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b156fc-4565-4d7f-86be-1b99d1164545 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:59.306576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 133/273] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:09:59 2022] GET /identity/v3/role_assignments?user.id=aeaea0c2d8494dfab06bfd3933c13ab0&scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f&role.id=753f8b35d5f0458d8f62d0ad6e28033f => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:59.431446 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3c673-3267-44f7-b710-c215bb325e6c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:59.439459 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e23918-9ec9-439f-92e9-831ac0ea8ace None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:59.439800 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e23918-9ec9-439f-92e9-831ac0ea8ace None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:59.440046 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e23918-9ec9-439f-92e9-831ac0ea8ace None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:59.442298 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 134/274] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:59.448296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa24a0-dfd8-4090-ae5d-9e234798ed75 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:59.448601 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa24a0-dfd8-4090-ae5d-9e234798ed75 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:59.448834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa24a0-dfd8-4090-ae5d-9e234798ed75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:59.471715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-bcaa24a0-dfd8-4090-ae5d-9e234798ed75 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:59.472373 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-bcaa24a0-dfd8-4090-ae5d-9e234798ed75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:59.504170 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:59.602054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 135/275] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:09:59.602594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208c5c7-790e-4279-92b1-93ada53eef4e None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:59.603011 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208c5c7-790e-4279-92b1-93ada53eef4e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:59.603289 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208c5c7-790e-4279-92b1-93ada53eef4e None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:59.609069 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 136/276] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:09:59.612314 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15b4a2-a113-4d3d-b5bf-27d742dcbab1 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:59.612590 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15b4a2-a113-4d3d-b5bf-27d742dcbab1 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:59.612857 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15b4a2-a113-4d3d-b5bf-27d742dcbab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:59.624804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3c673-3267-44f7-b710-c215bb325e6c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:59.634811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:59.637855 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-1b15b4a2-a113-4d3d-b5bf-27d742dcbab1 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:09:59.639150 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-1b15b4a2-a113-4d3d-b5bf-27d742dcbab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:09:59.684874 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b7fc2-5645-4f5f-908d-c6044fac8d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:59.685517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b7fc2-5645-4f5f-908d-c6044fac8d68 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:59.685880 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b7fc2-5645-4f5f-908d-c6044fac8d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:59.686206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b7fc2-5645-4f5f-908d-c6044fac8d68 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:59.698744 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b7fc2-5645-4f5f-908d-c6044fac8d68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:09:59.705617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-4a4b7fc2-5645-4f5f-908d-c6044fac8d68 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:09:59.705617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:09:59.705617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:09:59.705617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:09:59.705617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:09:59.705617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:09:59.705617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:09:59.705617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:09:59.705617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:09:59.705617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:09:59.705617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:09:59.705617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:09:59.705617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:09:59.705617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:09:59.705617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:09:59.705617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:09:59.706607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:09:59.706607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:09:59.706607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:09:59.706607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:09:59.706607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:09:59.706607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:09:59.706607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:09:59.706607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:09:59.706607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:09:59.706607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:09:59.706607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:09:59.706607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:09:59.706607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:09:59.706607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:09:59.706607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:09:59.706607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:09:59.707408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:09:59.707408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:09:59.707408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:09:59.707408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:09:59.707408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:09:59.707408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:09:59.707408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:09:59.707408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:09:59.707408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:09:59.707408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:09:59.707408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:09:59.707408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:09:59.707408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 141/277] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:09:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:09:59.708124 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b7fc2-5645-4f5f-908d-c6044fac8d68 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:59.735266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 137/278] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:09:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:09:59.736332 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15b4a2-a113-4d3d-b5bf-27d742dcbab1 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:59.812485 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b7fc2-5645-4f5f-908d-c6044fac8d68 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:59.823805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:59.849404 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3b6f5-87f5-416e-9695-5b249cc97a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:59.849730 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15b4a2-a113-4d3d-b5bf-27d742dcbab1 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:09:59.850036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3b6f5-87f5-416e-9695-5b249cc97a6b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:59.850315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3b6f5-87f5-416e-9695-5b249cc97a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:59.850585 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3b6f5-87f5-416e-9695-5b249cc97a6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:59.851829 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c3b6f5-87f5-416e-9695-5b249cc97a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:59.853426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c3b6f5-87f5-416e-9695-5b249cc97a6b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:59.855600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 142/279] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:09:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:59.861160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3b6f5-87f5-416e-9695-5b249cc97a6b None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:59.887182 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867244c-6ef0-4c1f-b121-e85b29de6d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:09:59.888270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867244c-6ef0-4c1f-b121-e85b29de6d15 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:09:59.889009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867244c-6ef0-4c1f-b121-e85b29de6d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:09:59.889750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867244c-6ef0-4c1f-b121-e85b29de6d15 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:09:59.890595 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d867244c-6ef0-4c1f-b121-e85b29de6d15 None admin] RBAC: Adding query filter params (name=member) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:09:59.891067 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d867244c-6ef0-4c1f-b121-e85b29de6d15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:09:59.892747 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d867244c-6ef0-4c1f-b121-e85b29de6d15 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:09:59.902383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 138/280] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:09:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:09:59.903526 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:09:59.910259 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867244c-6ef0-4c1f-b121-e85b29de6d15 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:09:59.968686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3b6f5-87f5-416e-9695-5b249cc97a6b None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:00.004059 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1716e8-04ae-4a8f-8a67-c259341d1608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:00.004584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1716e8-04ae-4a8f-8a67-c259341d1608 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:00.004860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1716e8-04ae-4a8f-8a67-c259341d1608 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:00.005127 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1716e8-04ae-4a8f-8a67-c259341d1608 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:00.005774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1716e8-04ae-4a8f-8a67-c259341d1608 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:00.007201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1716e8-04ae-4a8f-8a67-c259341d1608 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:00.014490 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867244c-6ef0-4c1f-b121-e85b29de6d15 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:00.028786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 143/281] 10.222.0.81 () {60 vars in 1132 bytes} [Mon Sep 19 11:09:59 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:10:00.036911 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1716e8-04ae-4a8f-8a67-c259341d1608 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:00.053106 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a218a-5fb2-4e8d-92c0-8ada436d30af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:00.053764 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a218a-5fb2-4e8d-92c0-8ada436d30af None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:00.054025 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a218a-5fb2-4e8d-92c0-8ada436d30af None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:00.054287 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a218a-5fb2-4e8d-92c0-8ada436d30af None admin] PATH_INFO: `/v3/users/3210dc50b8e4456eae3c11cb275b18f6` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:00.066865 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:00.070825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a218a-5fb2-4e8d-92c0-8ada436d30af None admin] RBAC: Authorizing `identity:get_user(user_id=3210dc50b8e4456eae3c11cb275b18f6)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:00.072647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a218a-5fb2-4e8d-92c0-8ada436d30af None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:00.074681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 139/282] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:09:59 2022] GET /identity/v3/users/3210dc50b8e4456eae3c11cb275b18f6 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:00.080743 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a218a-5fb2-4e8d-92c0-8ada436d30af None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:00.145976 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:00.157773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1716e8-04ae-4a8f-8a67-c259341d1608 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:00.196259 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40c928-5698-4c97-93a8-ab327a2a52eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:00.196928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40c928-5698-4c97-93a8-ab327a2a52eb None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:00.197211 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40c928-5698-4c97-93a8-ab327a2a52eb None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:00.198035 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40c928-5698-4c97-93a8-ab327a2a52eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:00.198392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a218a-5fb2-4e8d-92c0-8ada436d30af None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:00.211118 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40c928-5698-4c97-93a8-ab327a2a52eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:00.217343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-db40c928-5698-4c97-93a8-ab327a2a52eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:10:00.217343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:00.217343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:00.217343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:00.217343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:00.217343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:00.217343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:00.217343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:00.217343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:00.217343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:00.217343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:00.217343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:00.217343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:00.217343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:00.217343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:00.217343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:00.218252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:00.218252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:00.218252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:00.218252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:00.218252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:00.218252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:00.218252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:00.218252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:00.218252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:00.218252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:00.218252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:00.218252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:00.218252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:00.218252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:00.218252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:00.218252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:00.219063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:00.219063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:00.219063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:00.219063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:00.219063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:00.219063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:00.219063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:00.219063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:00.219063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:00.219063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:00.219063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:10:00.219063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:00.219063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 144/283] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:00.220242 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40c928-5698-4c97-93a8-ab327a2a52eb None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:00.249179 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bd6fd-5018-4129-855c-f3ac650e4ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:00.249179 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bd6fd-5018-4129-855c-f3ac650e4ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:00.249179 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bd6fd-5018-4129-855c-f3ac650e4ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:00.249621 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bd6fd-5018-4129-855c-f3ac650e4ec1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:00.262668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2bd6fd-5018-4129-855c-f3ac650e4ec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:00.268443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-fd2bd6fd-5018-4129-855c-f3ac650e4ec1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:10:00.268443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:00.268443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:00.268443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:00.268443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:00.268443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:00.268443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:00.268443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:00.268443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:00.268443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:00.268443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:00.268443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:00.268443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:00.268443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:00.268443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:00.268443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:00.269338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:00.269338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:00.269338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:00.269338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:00.269338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:00.269338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:00.269338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:00.269338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:00.269338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:00.269338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:00.269338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:00.269338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:00.269338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:00.269338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:00.269338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:00.269338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:00.270140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:00.270140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:00.270140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:00.270140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:00.270140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:00.270140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:00.270140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:00.270140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:00.270140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:00.270140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:00.270140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:10:00.270140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:00.270140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 140/284] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:00.270788 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bd6fd-5018-4129-855c-f3ac650e4ec1 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:00.345047 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40c928-5698-4c97-93a8-ab327a2a52eb None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:00.346738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:00.386540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e37ea-26ae-4974-ac1e-6546eb4b2c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:00.387149 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e37ea-26ae-4974-ac1e-6546eb4b2c5b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:00.387424 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e37ea-26ae-4974-ac1e-6546eb4b2c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:00.387707 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e37ea-26ae-4974-ac1e-6546eb4b2c5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:00.388471 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e37ea-26ae-4974-ac1e-6546eb4b2c5b None admin] RBAC: Adding query filter params (role.id=0c30c75e227c416da6e4814bce700e8d, user.id=3210dc50b8e4456eae3c11cb275b18f6, scope.system=all) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:00.388782 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e37ea-26ae-4974-ac1e-6546eb4b2c5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:00.390240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e37ea-26ae-4974-ac1e-6546eb4b2c5b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:00.396260 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 145/285] 10.222.0.81 () {56 vars in 1308 bytes} [Mon Sep 19 11:10:00 2022] GET /identity/v3/role_assignments?user.id=3210dc50b8e4456eae3c11cb275b18f6&scope.system=all&role.id=0c30c75e227c416da6e4814bce700e8d => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:00.397061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e37ea-26ae-4974-ac1e-6546eb4b2c5b None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:00.402799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bd6fd-5018-4129-855c-f3ac650e4ec1 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:00.415099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:00.455571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0904007-8422-4f0d-802e-0a4141769a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:00.456709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0904007-8422-4f0d-802e-0a4141769a8b None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:00.457278 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0904007-8422-4f0d-802e-0a4141769a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:00.457849 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0904007-8422-4f0d-802e-0a4141769a8b None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:00.459489 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0904007-8422-4f0d-802e-0a4141769a8b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:00.460334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0904007-8422-4f0d-802e-0a4141769a8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:00.462474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0904007-8422-4f0d-802e-0a4141769a8b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:00.475283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 141/286] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:10:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:00.481050 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0904007-8422-4f0d-802e-0a4141769a8b None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:00.512571 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e37ea-26ae-4974-ac1e-6546eb4b2c5b None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:00.584457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcc1b8-98a7-4d39-a796-feced6e9a027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:00.584978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcc1b8-98a7-4d39-a796-feced6e9a027 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:00.585258 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcc1b8-98a7-4d39-a796-feced6e9a027 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:00.585533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcc1b8-98a7-4d39-a796-feced6e9a027 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:00.586214 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcc1b8-98a7-4d39-a796-feced6e9a027 None admin] RBAC: Adding query filter params (name=member) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:00.586521 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcc1b8-98a7-4d39-a796-feced6e9a027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:00.587988 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcc1b8-98a7-4d39-a796-feced6e9a027 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:00.606848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 146/287] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:10:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:00.616879 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcc1b8-98a7-4d39-a796-feced6e9a027 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:00.670526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0904007-8422-4f0d-802e-0a4141769a8b None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:00.700105 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:00.724247 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:00.744162 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd485512-9ffd-4c29-98af-b26f912bdf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:00.745354 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd485512-9ffd-4c29-98af-b26f912bdf04 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:00.745633 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd485512-9ffd-4c29-98af-b26f912bdf04 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:00.745898 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd485512-9ffd-4c29-98af-b26f912bdf04 None admin] PATH_INFO: `/v3/users/3dc65c481dd34193b57064c161c65f81` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:00.748162 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd485512-9ffd-4c29-98af-b26f912bdf04 None admin] RBAC: Authorizing `identity:get_user(user_id=3dc65c481dd34193b57064c161c65f81)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:00.749661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd485512-9ffd-4c29-98af-b26f912bdf04 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:00.752760 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 142/288] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:10:00 2022] GET /identity/v3/users/3dc65c481dd34193b57064c161c65f81 => generated 297 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:00.756706 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd485512-9ffd-4c29-98af-b26f912bdf04 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:00.764632 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcc1b8-98a7-4d39-a796-feced6e9a027 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:00.811846 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7faa3f-ceb9-47e1-9829-616d0d3219a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:00.812478 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7faa3f-ceb9-47e1-9829-616d0d3219a0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:00.812754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7faa3f-ceb9-47e1-9829-616d0d3219a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:00.813022 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7faa3f-ceb9-47e1-9829-616d0d3219a0 None admin] PATH_INFO: `/v3/users/8db1efb838bc4804be34eb46ec2a001b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:00.818261 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7faa3f-ceb9-47e1-9829-616d0d3219a0 None admin] RBAC: Authorizing `identity:get_user(user_id=8db1efb838bc4804be34eb46ec2a001b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:00.819744 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7faa3f-ceb9-47e1-9829-616d0d3219a0 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:00.821885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 147/289] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:10:00 2022] GET /identity/v3/users/8db1efb838bc4804be34eb46ec2a001b => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:00.829116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7faa3f-ceb9-47e1-9829-616d0d3219a0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:00.921690 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd485512-9ffd-4c29-98af-b26f912bdf04 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:00.959504 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef102e6-d385-45aa-b4bc-f79148949a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:00.960136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef102e6-d385-45aa-b4bc-f79148949a75 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:00.960412 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef102e6-d385-45aa-b4bc-f79148949a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:00.960680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef102e6-d385-45aa-b4bc-f79148949a75 None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:00.961814 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef102e6-d385-45aa-b4bc-f79148949a75 None admin] RBAC: Authorizing `identity:get_project(project_id=8eb05b28107f4365a8f3ff9b192ce72f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:00.963291 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef102e6-d385-45aa-b4bc-f79148949a75 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:00.971366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 143/290] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:10:00 2022] GET /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:00.977524 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef102e6-d385-45aa-b4bc-f79148949a75 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:01.045626 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:01.048906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7faa3f-ceb9-47e1-9829-616d0d3219a0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:01.094844 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef102e6-d385-45aa-b4bc-f79148949a75 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:01.128601 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6bab2-0b81-4331-9c89-53bfa97f4cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:01.129363 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6bab2-0b81-4331-9c89-53bfa97f4cfb None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:01.129741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6bab2-0b81-4331-9c89-53bfa97f4cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:01.130044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6bab2-0b81-4331-9c89-53bfa97f4cfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:01.131441 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4ac66-b49e-4647-b313-4f2f699ff93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:01.132124 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4ac66-b49e-4647-b313-4f2f699ff93b None admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:01.132401 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4ac66-b49e-4647-b313-4f2f699ff93b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:01.132672 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4ac66-b49e-4647-b313-4f2f699ff93b None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f/users/3dc65c481dd34193b57064c161c65f81/roles/967fbd121d5e4d65b135451eaf2c03d8` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:01.134597 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed4ac66-b49e-4647-b313-4f2f699ff93b None admin] RBAC: Authorizing `identity:create_grant(project_id=8eb05b28107f4365a8f3ff9b192ce72f, user_id=3dc65c481dd34193b57064c161c65f81, role_id=967fbd121d5e4d65b135451eaf2c03d8)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:01.136086 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed4ac66-b49e-4647-b313-4f2f699ff93b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:01.156060 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 144/291] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:10:00 2022] PUT /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f/users/3dc65c481dd34193b57064c161c65f81/roles/967fbd121d5e4d65b135451eaf2c03d8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:10:01.164109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:01.278685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b6bab2-0b81-4331-9c89-53bfa97f4cfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:01.280135 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b6bab2-0b81-4331-9c89-53bfa97f4cfb None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:01.408006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 148/292] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:10:01.413615 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4ac66-b49e-4647-b313-4f2f699ff93b None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:01.519269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4ac66-b49e-4647-b313-4f2f699ff93b None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:01.537056 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:01.555481 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d85d5-6fa1-4f05-8274-71db28d947a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:01.556141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d85d5-6fa1-4f05-8274-71db28d947a9 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:01.556595 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d85d5-6fa1-4f05-8274-71db28d947a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:01.557487 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d85d5-6fa1-4f05-8274-71db28d947a9 None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:01.558970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d85d5-6fa1-4f05-8274-71db28d947a9 None admin] RBAC: Authorizing `identity:get_project(project_id=8eb05b28107f4365a8f3ff9b192ce72f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:01.560599 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d85d5-6fa1-4f05-8274-71db28d947a9 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:01.562475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 145/293] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:10:01 2022] GET /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:01.568276 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6bab2-0b81-4331-9c89-53bfa97f4cfb None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:01.632036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:01.709748 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6bab2-0b81-4331-9c89-53bfa97f4cfb None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:01.753669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410ec24-1474-4543-bd47-6347701fc7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:01.754278 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410ec24-1474-4543-bd47-6347701fc7d8 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:01.754555 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410ec24-1474-4543-bd47-6347701fc7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:01.754827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410ec24-1474-4543-bd47-6347701fc7d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:01.755552 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b410ec24-1474-4543-bd47-6347701fc7d8 None admin] RBAC: Adding query filter params (role.id=0c30c75e227c416da6e4814bce700e8d, scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f, user.id=8db1efb838bc4804be34eb46ec2a001b) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:01.755882 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b410ec24-1474-4543-bd47-6347701fc7d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:01.757338 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b410ec24-1474-4543-bd47-6347701fc7d8 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:01.764312 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 149/294] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:10:01 2022] GET /identity/v3/role_assignments?user.id=8db1efb838bc4804be34eb46ec2a001b&scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f&role.id=0c30c75e227c416da6e4814bce700e8d => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:02.673303 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af398a29-c439-4046-a453-c3f421aa2a37 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:02.674135 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af398a29-c439-4046-a453-c3f421aa2a37 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:02.674617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af398a29-c439-4046-a453-c3f421aa2a37 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:02.676476 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 146/295] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:02.682084 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5691a8-b9b1-4e8c-a2cf-e66b6f808fda None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:02.682941 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5691a8-b9b1-4e8c-a2cf-e66b6f808fda None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:02.683416 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5691a8-b9b1-4e8c-a2cf-e66b6f808fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:02.703683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-1f5691a8-b9b1-4e8c-a2cf-e66b6f808fda None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:02.705320 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1f5691a8-b9b1-4e8c-a2cf-e66b6f808fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:02.783873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 150/296] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:02.845116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66841cf4-5475-4385-ad14-2079cd6d5764 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:02.845752 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66841cf4-5475-4385-ad14-2079cd6d5764 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:02.846224 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66841cf4-5475-4385-ad14-2079cd6d5764 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:02.847827 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 147/297] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:02.854295 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeed40d-0974-4f69-89c5-edd872a67c83 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:02.854623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeed40d-0974-4f69-89c5-edd872a67c83 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:02.854846 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeed40d-0974-4f69-89c5-edd872a67c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:02.875901 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-aaeed40d-0974-4f69-89c5-edd872a67c83 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:02.877136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-aaeed40d-0974-4f69-89c5-edd872a67c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:02.907993 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:02.946410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 151/298] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:02.975210 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c376b-7e44-4198-8b30-1173692c2d6b None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:02.975525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c376b-7e44-4198-8b30-1173692c2d6b None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:02.975789 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c376b-7e44-4198-8b30-1173692c2d6b None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:02.977252 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 148/299] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:02.983050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b165880-463b-493d-8f4f-61bd43e498c2 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:02.983339 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b165880-463b-493d-8f4f-61bd43e498c2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:02.983609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b165880-463b-493d-8f4f-61bd43e498c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:03.002199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-0b165880-463b-493d-8f4f-61bd43e498c2 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:03.003656 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0b165880-463b-493d-8f4f-61bd43e498c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:03.075965 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 152/300] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:03.086968 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c376b-7e44-4198-8b30-1173692c2d6b None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:03.200701 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c376b-7e44-4198-8b30-1173692c2d6b None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:03.234987 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f8c5e-6f86-4f14-88ff-a406b4efcae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:03.235663 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f8c5e-6f86-4f14-88ff-a406b4efcae2 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:03.235967 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f8c5e-6f86-4f14-88ff-a406b4efcae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:03.236236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f8c5e-6f86-4f14-88ff-a406b4efcae2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:03.247825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f8c5e-6f86-4f14-88ff-a406b4efcae2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:03.252598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-2f8f8c5e-6f86-4f14-88ff-a406b4efcae2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:10:03.252598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:03.252598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:03.252598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:03.252598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:03.252598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:03.252598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:03.252598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:03.252598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:03.252598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:03.252598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:03.252598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:03.252598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:03.252598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:03.252598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:03.252598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:03.253498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:03.253498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:03.253498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:03.253498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:03.253498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:03.253498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:03.253498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:03.253498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:03.253498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:03.253498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:03.253498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:03.253498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:03.253498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:03.253498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:03.253498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:03.253498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:03.254290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:03.254290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:03.254290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:03.254290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:03.254290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:03.254290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:03.254290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:03.254290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:03.254290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:03.254290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:03.254290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:10:03.254290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:03.254290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 149/301] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:03.259919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b165880-463b-493d-8f4f-61bd43e498c2 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:03.260259 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f8c5e-6f86-4f14-88ff-a406b4efcae2 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:03.319555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:03.364482 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:03.368735 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b165880-463b-493d-8f4f-61bd43e498c2 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:03.382750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f8c5e-6f86-4f14-88ff-a406b4efcae2 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:03.407361 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcef7b4-bc97-43b3-969e-0d264a516d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:03.408002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcef7b4-bc97-43b3-969e-0d264a516d68 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:03.408289 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcef7b4-bc97-43b3-969e-0d264a516d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:03.408560 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcef7b4-bc97-43b3-969e-0d264a516d68 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:03.409231 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcef7b4-bc97-43b3-969e-0d264a516d68 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:03.409543 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcef7b4-bc97-43b3-969e-0d264a516d68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:03.410915 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcef7b4-bc97-43b3-969e-0d264a516d68 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:03.420273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 153/302] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:10:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:03.421378 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcef7b4-bc97-43b3-969e-0d264a516d68 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:03.426811 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9779b8-f246-40ce-80ae-3842a6a4caba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:03.427449 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9779b8-f246-40ce-80ae-3842a6a4caba None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:03.427804 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9779b8-f246-40ce-80ae-3842a6a4caba None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:03.428075 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9779b8-f246-40ce-80ae-3842a6a4caba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:03.429174 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9779b8-f246-40ce-80ae-3842a6a4caba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:03.431110 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9779b8-f246-40ce-80ae-3842a6a4caba None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:03.433882 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 150/303] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:03.435160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9779b8-f246-40ce-80ae-3842a6a4caba None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:03.526822 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcef7b4-bc97-43b3-969e-0d264a516d68 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:03.538048 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9779b8-f246-40ce-80ae-3842a6a4caba None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:03.552387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:03.561209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d9713-9a16-443f-b0e6-4f097a455566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:03.561853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d9713-9a16-443f-b0e6-4f097a455566 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:03.562135 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d9713-9a16-443f-b0e6-4f097a455566 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:03.562411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d9713-9a16-443f-b0e6-4f097a455566 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:03.574569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d9713-9a16-443f-b0e6-4f097a455566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:03.578204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cab128-eb90-497d-84f4-dcf682a70817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:03.579372 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cab128-eb90-497d-84f4-dcf682a70817 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:03.579872 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cab128-eb90-497d-84f4-dcf682a70817 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:03.580323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cab128-eb90-497d-84f4-dcf682a70817 None admin] PATH_INFO: `/v3/users/1274b71a5bbe4b2d91f97043bb735e32` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:03.580426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-bd3d9713-9a16-443f-b0e6-4f097a455566 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:10:03.580426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:03.580426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:03.580426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:03.580426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:03.580426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:03.580426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:03.580426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:03.580426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:03.580426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:03.580426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:03.580426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:03.580426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:03.580426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:03.580426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:03.580426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:03.581391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:03.581391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:03.581391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:03.581391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:03.581391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:03.581391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:03.581391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:03.581391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:03.581391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:03.581391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:03.581391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:03.581391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:03.581391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:03.581391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:03.581391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:03.581391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:03.582319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:03.582319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:03.582319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:03.582319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:03.582319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:03.582319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:03.582319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:03.582319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:03.582319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:03.582319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:03.582319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:10:03.582319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:03.582319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 154/304] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:03.587355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d9713-9a16-443f-b0e6-4f097a455566 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:03.600651 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cab128-eb90-497d-84f4-dcf682a70817 None admin] RBAC: Authorizing `identity:get_user(user_id=1274b71a5bbe4b2d91f97043bb735e32)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:03.602457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cab128-eb90-497d-84f4-dcf682a70817 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:03.604917 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 151/305] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:10:03 2022] GET /identity/v3/users/1274b71a5bbe4b2d91f97043bb735e32 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:03.605509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cab128-eb90-497d-84f4-dcf682a70817 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:03.616274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:03.704441 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d9713-9a16-443f-b0e6-4f097a455566 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:03.717355 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cab128-eb90-497d-84f4-dcf682a70817 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:03.738306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac8015-d73f-4fa7-a099-00b6ed63947d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:03.738909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac8015-d73f-4fa7-a099-00b6ed63947d None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:03.739187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac8015-d73f-4fa7-a099-00b6ed63947d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:03.739458 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac8015-d73f-4fa7-a099-00b6ed63947d None admin] PATH_INFO: `/v3/groups` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:03.740116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac8015-d73f-4fa7-a099-00b6ed63947d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:03.741566 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac8015-d73f-4fa7-a099-00b6ed63947d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:03.749638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34610732-f6a8-4f11-ba1f-d7e27a20e82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:03.750324 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34610732-f6a8-4f11-ba1f-d7e27a20e82f None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:03.750598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34610732-f6a8-4f11-ba1f-d7e27a20e82f None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:03.750879 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34610732-f6a8-4f11-ba1f-d7e27a20e82f None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:03.751531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34610732-f6a8-4f11-ba1f-d7e27a20e82f None admin] RBAC: Adding query filter params (name=member) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:03.751929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34610732-f6a8-4f11-ba1f-d7e27a20e82f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:03.753271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34610732-f6a8-4f11-ba1f-d7e27a20e82f None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:03.756834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 155/306] 10.222.0.81 () {60 vars in 1134 bytes} [Mon Sep 19 11:10:03 2022] POST /identity/v3/groups => generated 229 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:10:03.757631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac8015-d73f-4fa7-a099-00b6ed63947d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:03.767244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 152/307] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:10:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:03.768670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54db23-d3df-476f-84d8-43910895ea5c None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:03.768962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54db23-d3df-476f-84d8-43910895ea5c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:03.769239 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54db23-d3df-476f-84d8-43910895ea5c None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:03.770932 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 153/308] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:03.773435 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54db23-d3df-476f-84d8-43910895ea5c None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:03.810389 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:03.874588 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:03.894579 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54db23-d3df-476f-84d8-43910895ea5c None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:03.897689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac8015-d73f-4fa7-a099-00b6ed63947d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:03.934354 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30531622-9a5d-4020-b644-397422d8b740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:03.935859 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30531622-9a5d-4020-b644-397422d8b740 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:03.936143 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30531622-9a5d-4020-b644-397422d8b740 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:03.936414 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30531622-9a5d-4020-b644-397422d8b740 None admin] PATH_INFO: `/v3/users/3210dc50b8e4456eae3c11cb275b18f6` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:03.937544 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30531622-9a5d-4020-b644-397422d8b740 None admin] RBAC: Authorizing `identity:get_user(user_id=3210dc50b8e4456eae3c11cb275b18f6)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:03.937841 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb0d35-7553-4051-8eaf-54b6db18bdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:03.938634 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb0d35-7553-4051-8eaf-54b6db18bdaf None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:03.938913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb0d35-7553-4051-8eaf-54b6db18bdaf None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:03.939014 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30531622-9a5d-4020-b644-397422d8b740 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:03.939185 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb0d35-7553-4051-8eaf-54b6db18bdaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:03.939946 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb0d35-7553-4051-8eaf-54b6db18bdaf None admin] RBAC: Adding query filter params (role.id=967fbd121d5e4d65b135451eaf2c03d8, user.id=1274b71a5bbe4b2d91f97043bb735e32, scope.system=all) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:03.940257 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb0d35-7553-4051-8eaf-54b6db18bdaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:03.940788 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 154/309] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:10:03 2022] GET /identity/v3/users/3210dc50b8e4456eae3c11cb275b18f6 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:03.941691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb0d35-7553-4051-8eaf-54b6db18bdaf None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:03.941786 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cbe2d-2d5a-4bd4-8090-bd6fcd5d85a7 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:03.942002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cbe2d-2d5a-4bd4-8090-bd6fcd5d85a7 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:03.942281 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cbe2d-2d5a-4bd4-8090-bd6fcd5d85a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:03.947806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 156/310] 10.222.0.81 () {56 vars in 1308 bytes} [Mon Sep 19 11:10:03 2022] GET /identity/v3/role_assignments?user.id=1274b71a5bbe4b2d91f97043bb735e32&scope.system=all&role.id=967fbd121d5e4d65b135451eaf2c03d8 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:03.948193 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb0d35-7553-4051-8eaf-54b6db18bdaf None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:03.961502 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-5d8cbe2d-2d5a-4bd4-8090-bd6fcd5d85a7 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:03.962746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5d8cbe2d-2d5a-4bd4-8090-bd6fcd5d85a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:04.056151 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 155/311] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:04.087340 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb0d35-7553-4051-8eaf-54b6db18bdaf None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:04.125437 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dd138-b71e-4672-8628-924f20908d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:04.126105 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dd138-b71e-4672-8628-924f20908d22 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:04.126383 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dd138-b71e-4672-8628-924f20908d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:04.126657 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dd138-b71e-4672-8628-924f20908d22 None admin] PATH_INFO: `/v3/system/users/3210dc50b8e4456eae3c11cb275b18f6/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:04.128151 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5dd138-b71e-4672-8628-924f20908d22 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3210dc50b8e4456eae3c11cb275b18f6, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:04.129558 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5dd138-b71e-4672-8628-924f20908d22 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:04.147863 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 157/312] 10.222.0.81 () {58 vars in 1334 bytes} [Mon Sep 19 11:10:03 2022] PUT /identity/v3/system/users/3210dc50b8e4456eae3c11cb275b18f6/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:10:04.329935 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8cbe2d-2d5a-4bd4-8090-bd6fcd5d85a7 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:04.360915 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:04.432320 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8cbe2d-2d5a-4bd4-8090-bd6fcd5d85a7 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:04.471239 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303af45-f48e-447c-bfd7-5e4acd475423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:04.471944 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303af45-f48e-447c-bfd7-5e4acd475423 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:04.472224 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303af45-f48e-447c-bfd7-5e4acd475423 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:04.472496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303af45-f48e-447c-bfd7-5e4acd475423 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:04.483789 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303af45-f48e-447c-bfd7-5e4acd475423 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:04.488623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-4303af45-f48e-447c-bfd7-5e4acd475423 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:10:04.488623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:04.488623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:04.488623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:04.488623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:04.488623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:04.488623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:04.488623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:04.488623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:04.488623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:04.488623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:04.488623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:04.488623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:04.488623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:04.488623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:04.488623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:04.489514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:04.489514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:04.489514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:04.489514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:04.489514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:04.489514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:04.489514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:04.489514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:04.489514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:04.489514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:04.489514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:04.489514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:04.489514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:04.489514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:04.489514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:04.489514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:04.490316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:04.490316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:04.490316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:04.490316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:04.490316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:04.490316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:04.490316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:04.490316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:04.490316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:04.490316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:04.490316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:10:04.490316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:04.490316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 156/313] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:04.495595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dd138-b71e-4672-8628-924f20908d22 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:04.512141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:04.611228 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dd138-b71e-4672-8628-924f20908d22 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:04.647078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1472a87-083f-4edc-955d-1ce76eb9d39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:04.647723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1472a87-083f-4edc-955d-1ce76eb9d39e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:04.648039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1472a87-083f-4edc-955d-1ce76eb9d39e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:04.648372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1472a87-083f-4edc-955d-1ce76eb9d39e None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:04.649098 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1472a87-083f-4edc-955d-1ce76eb9d39e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:04.649405 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1472a87-083f-4edc-955d-1ce76eb9d39e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:04.651466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1472a87-083f-4edc-955d-1ce76eb9d39e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:04.660880 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 158/314] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:10:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:04.671175 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303af45-f48e-447c-bfd7-5e4acd475423 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:04.770028 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:04.780474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303af45-f48e-447c-bfd7-5e4acd475423 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:04.816406 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b7c86-2f0d-49cd-89df-040f3f723f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:04.817025 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b7c86-2f0d-49cd-89df-040f3f723f66 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:04.817302 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b7c86-2f0d-49cd-89df-040f3f723f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:04.817571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b7c86-2f0d-49cd-89df-040f3f723f66 None admin] PATH_INFO: `/v3/users/3dc65c481dd34193b57064c161c65f81` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:04.818704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b7c86-2f0d-49cd-89df-040f3f723f66 None admin] RBAC: Authorizing `identity:get_user(user_id=3dc65c481dd34193b57064c161c65f81)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:04.820176 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b7c86-2f0d-49cd-89df-040f3f723f66 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:04.821994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 157/315] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:10:04 2022] GET /identity/v3/users/3dc65c481dd34193b57064c161c65f81 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:04.827581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1472a87-083f-4edc-955d-1ce76eb9d39e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:04.903883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:04.936106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1472a87-083f-4edc-955d-1ce76eb9d39e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:04.971638 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2963a-f901-4705-91d1-7128069b0fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:04.972334 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2963a-f901-4705-91d1-7128069b0fcd None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:04.972716 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2963a-f901-4705-91d1-7128069b0fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:04.972997 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2963a-f901-4705-91d1-7128069b0fcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:05.042114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb2963a-f901-4705-91d1-7128069b0fcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:05.043666 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb2963a-f901-4705-91d1-7128069b0fcd None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:05.116212 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:05.154354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 159/316] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:10:05.155939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b7c86-2f0d-49cd-89df-040f3f723f66 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:05.259463 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b7c86-2f0d-49cd-89df-040f3f723f66 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:05.300646 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd35cb4-5a12-42c9-9d7f-ae0060d61f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:05.301275 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd35cb4-5a12-42c9-9d7f-ae0060d61f77 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:05.301604 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd35cb4-5a12-42c9-9d7f-ae0060d61f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:05.301880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd35cb4-5a12-42c9-9d7f-ae0060d61f77 None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:05.302998 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd35cb4-5a12-42c9-9d7f-ae0060d61f77 None admin] RBAC: Authorizing `identity:get_project(project_id=8eb05b28107f4365a8f3ff9b192ce72f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:05.304462 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd35cb4-5a12-42c9-9d7f-ae0060d61f77 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:05.311704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 158/317] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:10:05 2022] GET /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:05.313155 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2963a-f901-4705-91d1-7128069b0fcd None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:05.415884 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2963a-f901-4705-91d1-7128069b0fcd None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:05.452029 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0984d-cebd-4317-a44b-275b0ab6e07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:05.452637 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0984d-cebd-4317-a44b-275b0ab6e07c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:05.452919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0984d-cebd-4317-a44b-275b0ab6e07c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:05.453189 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0984d-cebd-4317-a44b-275b0ab6e07c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:05.454177 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0984d-cebd-4317-a44b-275b0ab6e07c None admin] RBAC: Adding query filter params (role.id=967fbd121d5e4d65b135451eaf2c03d8, scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f, user.id=3dc65c481dd34193b57064c161c65f81) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:05.454484 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0984d-cebd-4317-a44b-275b0ab6e07c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:05.456030 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0984d-cebd-4317-a44b-275b0ab6e07c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:05.462359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 160/318] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:10:05 2022] GET /identity/v3/role_assignments?user.id=3dc65c481dd34193b57064c161c65f81&scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f&role.id=967fbd121d5e4d65b135451eaf2c03d8 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:06.118919 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0a742-5eed-497d-ad6a-cdd7ccf725d5 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:06.119512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0a742-5eed-497d-ad6a-cdd7ccf725d5 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:06.119512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0a742-5eed-497d-ad6a-cdd7ccf725d5 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:06.120886 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 159/319] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:06.126068 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98d195-cbf6-4cbb-8815-f4eaf72b81ba None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:06.126619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98d195-cbf6-4cbb-8815-f4eaf72b81ba None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:06.126619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98d195-cbf6-4cbb-8815-f4eaf72b81ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:06.146462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6c98d195-cbf6-4cbb-8815-f4eaf72b81ba None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:06.147674 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6c98d195-cbf6-4cbb-8815-f4eaf72b81ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:06.175855 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:06.199937 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc53a7a-6c2a-4d48-874c-3f85486951bf None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:06.200220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc53a7a-6c2a-4d48-874c-3f85486951bf None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:06.200495 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc53a7a-6c2a-4d48-874c-3f85486951bf None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:06.201946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 160/320] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:06.206744 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986ef43-87e3-4933-aef5-a1ae08c67426 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:06.207030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986ef43-87e3-4933-aef5-a1ae08c67426 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:06.207304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986ef43-87e3-4933-aef5-a1ae08c67426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:06.218921 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 161/321] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:06.225255 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-3986ef43-87e3-4933-aef5-a1ae08c67426 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:06.226481 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3986ef43-87e3-4933-aef5-a1ae08c67426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:06.247260 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:06.295303 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 161/322] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:10:06.426671 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d5e47-92ef-4ba5-a329-ce42b3212ce5 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:06.426982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d5e47-92ef-4ba5-a329-ce42b3212ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:06.427152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d5e47-92ef-4ba5-a329-ce42b3212ce5 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:06.428648 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 162/323] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:06.433729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cddd1a-67d7-426b-82fc-3ec516e5ea67 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:06.434018 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cddd1a-67d7-426b-82fc-3ec516e5ea67 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:06.434295 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cddd1a-67d7-426b-82fc-3ec516e5ea67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:06.452789 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-35cddd1a-67d7-426b-82fc-3ec516e5ea67 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:06.454288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-35cddd1a-67d7-426b-82fc-3ec516e5ea67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:06.488977 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d5e47-92ef-4ba5-a329-ce42b3212ce5 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:06.524999 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 162/324] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:06.564724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cddd1a-67d7-426b-82fc-3ec516e5ea67 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:06.595151 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d5e47-92ef-4ba5-a329-ce42b3212ce5 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:06.630447 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c9086-344f-4f15-b03b-149abf2683f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:06.631138 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c9086-344f-4f15-b03b-149abf2683f7 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:06.631417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c9086-344f-4f15-b03b-149abf2683f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:06.631792 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c9086-344f-4f15-b03b-149abf2683f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:06.632803 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c9086-344f-4f15-b03b-149abf2683f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:06.634313 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c9086-344f-4f15-b03b-149abf2683f7 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:06.636641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 163/325] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:06.642122 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c9086-344f-4f15-b03b-149abf2683f7 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:06.663376 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:06.678449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:06.684842 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cddd1a-67d7-426b-82fc-3ec516e5ea67 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:06.723110 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b75e6-2bf4-40ea-82d6-d957c32d7dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:06.723737 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b75e6-2bf4-40ea-82d6-d957c32d7dcc None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:06.724040 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b75e6-2bf4-40ea-82d6-d957c32d7dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:06.724307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b75e6-2bf4-40ea-82d6-d957c32d7dcc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:06.737019 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b75e6-2bf4-40ea-82d6-d957c32d7dcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:06.743262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-0a4b75e6-2bf4-40ea-82d6-d957c32d7dcc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:10:06.743262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:06.743262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:06.743262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:06.743262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:06.743262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:06.743262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:06.743262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:06.743262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:06.743262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:06.743262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:06.743262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:06.743262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:06.743262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:06.743262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:06.743262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:06.744801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:06.744801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:06.744801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:06.744801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:06.744801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:06.744801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:06.744801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:06.744801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:06.744801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:06.744801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:06.744801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:06.744801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:06.744801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:06.744801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:06.744801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:06.744801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:06.745622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:06.745622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:06.745622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:06.745622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:06.745622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:06.745622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:06.745622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:06.745622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:06.745622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:06.745622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:06.745622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:10:06.745622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:06.745622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 163/326] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:10:06.750770 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b75e6-2bf4-40ea-82d6-d957c32d7dcc None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:06.754050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c9086-344f-4f15-b03b-149abf2683f7 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:06.788710 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74e289-2ceb-4ea4-8dd9-b7e700f8af22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:06.789317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74e289-2ceb-4ea4-8dd9-b7e700f8af22 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:06.789596 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74e289-2ceb-4ea4-8dd9-b7e700f8af22 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:06.789867 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74e289-2ceb-4ea4-8dd9-b7e700f8af22 None admin] PATH_INFO: `/v3/groups` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:06.790557 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74e289-2ceb-4ea4-8dd9-b7e700f8af22 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:06.792111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74e289-2ceb-4ea4-8dd9-b7e700f8af22 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:06.808518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 164/327] 10.222.0.81 () {60 vars in 1134 bytes} [Mon Sep 19 11:10:06 2022] POST /identity/v3/groups => generated 226 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:10:06.809225 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74e289-2ceb-4ea4-8dd9-b7e700f8af22 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:06.868433 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b75e6-2bf4-40ea-82d6-d957c32d7dcc None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:06.909444 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ea014-4b8b-4c53-9088-6c4a4456bb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:06.910067 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ea014-4b8b-4c53-9088-6c4a4456bb29 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:06.910343 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ea014-4b8b-4c53-9088-6c4a4456bb29 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:06.910618 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ea014-4b8b-4c53-9088-6c4a4456bb29 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:06.910712 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:06.911298 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ea014-4b8b-4c53-9088-6c4a4456bb29 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:06.911607 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ea014-4b8b-4c53-9088-6c4a4456bb29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:06.913060 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ea014-4b8b-4c53-9088-6c4a4456bb29 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:06.922255 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 164/328] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:10:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:06.923640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74e289-2ceb-4ea4-8dd9-b7e700f8af22 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:06.927946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ea014-4b8b-4c53-9088-6c4a4456bb29 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:06.941147 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:06.967200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe925b-07de-4496-bfd1-4742603212b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:06.967888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe925b-07de-4496-bfd1-4742603212b0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:06.968175 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe925b-07de-4496-bfd1-4742603212b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:06.968452 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe925b-07de-4496-bfd1-4742603212b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:06.980316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe925b-07de-4496-bfd1-4742603212b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:06.986034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-cefe925b-07de-4496-bfd1-4742603212b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:10:06.986034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:06.986034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:06.986034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:06.986034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:06.986034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:06.986034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:06.986034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:06.986034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:06.986034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:06.986034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:06.986034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:06.986034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:06.986034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:06.986034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:06.986034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:06.986913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:06.986913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:06.986913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:06.986913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:06.986913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:06.986913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:06.986913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:06.986913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:06.986913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:06.986913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:06.986913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:06.986913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:06.986913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:06.986913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:06.986913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:06.986913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:06.987754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:06.987754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:06.987754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:06.987754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:06.987754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:06.987754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:06.987754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:06.987754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:06.987754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:06.987754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:06.987754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:10:06.987754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:06.987754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 165/329] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:06.993178 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe925b-07de-4496-bfd1-4742603212b0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:07.053659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ea014-4b8b-4c53-9088-6c4a4456bb29 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:07.100849 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475fa46-818a-4eb3-a278-f51c100c0ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:07.101489 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475fa46-818a-4eb3-a278-f51c100c0ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:07.101769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475fa46-818a-4eb3-a278-f51c100c0ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:07.102043 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475fa46-818a-4eb3-a278-f51c100c0ee7 None admin] PATH_INFO: `/v3/users/1274b71a5bbe4b2d91f97043bb735e32` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:07.104860 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475fa46-818a-4eb3-a278-f51c100c0ee7 None admin] RBAC: Authorizing `identity:get_user(user_id=1274b71a5bbe4b2d91f97043bb735e32)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:07.106307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475fa46-818a-4eb3-a278-f51c100c0ee7 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:07.108803 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 165/330] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:10:06 2022] GET /identity/v3/users/1274b71a5bbe4b2d91f97043bb735e32 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:07.116245 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475fa46-818a-4eb3-a278-f51c100c0ee7 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:07.125035 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe925b-07de-4496-bfd1-4742603212b0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:07.164516 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637fd18-f1bc-4a44-96d7-c9fbf2799c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:07.165148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637fd18-f1bc-4a44-96d7-c9fbf2799c9c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:07.165527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637fd18-f1bc-4a44-96d7-c9fbf2799c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:07.165810 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637fd18-f1bc-4a44-96d7-c9fbf2799c9c None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:07.167683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637fd18-f1bc-4a44-96d7-c9fbf2799c9c None admin] RBAC: Adding query filter params (name=member) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:07.169533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637fd18-f1bc-4a44-96d7-c9fbf2799c9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:07.169533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637fd18-f1bc-4a44-96d7-c9fbf2799c9c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:07.180575 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 166/331] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:10:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:07.186582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637fd18-f1bc-4a44-96d7-c9fbf2799c9c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:07.207217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:07.214813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:07.244750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475fa46-818a-4eb3-a278-f51c100c0ee7 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:07.296991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36e0b5-d72a-4948-85dd-20b924f38516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:07.297754 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36e0b5-d72a-4948-85dd-20b924f38516 None admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:07.298109 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36e0b5-d72a-4948-85dd-20b924f38516 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:07.298644 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36e0b5-d72a-4948-85dd-20b924f38516 None admin] PATH_INFO: `/v3/system/users/1274b71a5bbe4b2d91f97043bb735e32/roles/967fbd121d5e4d65b135451eaf2c03d8` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:07.300594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36e0b5-d72a-4948-85dd-20b924f38516 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1274b71a5bbe4b2d91f97043bb735e32, role_id=967fbd121d5e4d65b135451eaf2c03d8)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:07.302558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36e0b5-d72a-4948-85dd-20b924f38516 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:07.314709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 166/332] 10.222.0.81 () {58 vars in 1334 bytes} [Mon Sep 19 11:10:07 2022] PUT /identity/v3/system/users/1274b71a5bbe4b2d91f97043bb735e32/roles/967fbd121d5e4d65b135451eaf2c03d8 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:10:07.346568 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637fd18-f1bc-4a44-96d7-c9fbf2799c9c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:07.395134 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d292ba-4d95-4207-8e1b-1e08334f5df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:07.395806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d292ba-4d95-4207-8e1b-1e08334f5df3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:07.396091 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d292ba-4d95-4207-8e1b-1e08334f5df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:07.396362 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d292ba-4d95-4207-8e1b-1e08334f5df3 None admin] PATH_INFO: `/v3/users/3210dc50b8e4456eae3c11cb275b18f6` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:07.397636 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d292ba-4d95-4207-8e1b-1e08334f5df3 None admin] RBAC: Authorizing `identity:get_user(user_id=3210dc50b8e4456eae3c11cb275b18f6)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:07.399117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d292ba-4d95-4207-8e1b-1e08334f5df3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:07.401124 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 167/333] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:10:07 2022] GET /identity/v3/users/3210dc50b8e4456eae3c11cb275b18f6 => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:07.407068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36e0b5-d72a-4948-85dd-20b924f38516 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:07.528887 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36e0b5-d72a-4948-85dd-20b924f38516 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:07.566486 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5131dd-feb8-462f-a9ed-8347cea66559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:07.567146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5131dd-feb8-462f-a9ed-8347cea66559 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:07.567430 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5131dd-feb8-462f-a9ed-8347cea66559 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:07.567701 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5131dd-feb8-462f-a9ed-8347cea66559 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:07.568519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5131dd-feb8-462f-a9ed-8347cea66559 None admin] RBAC: Adding query filter params (role.id=0c30c75e227c416da6e4814bce700e8d, user.id=3210dc50b8e4456eae3c11cb275b18f6, scope.system=all) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:07.568828 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5131dd-feb8-462f-a9ed-8347cea66559 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:07.570279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5131dd-feb8-462f-a9ed-8347cea66559 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:07.576562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:07.586515 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 167/334] 10.222.0.81 () {56 vars in 1308 bytes} [Mon Sep 19 11:10:07 2022] GET /identity/v3/role_assignments?user.id=3210dc50b8e4456eae3c11cb275b18f6&scope.system=all&role.id=0c30c75e227c416da6e4814bce700e8d => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:09.223155 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5600f-9645-4eea-9d4c-950ae3f088c7 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:09.223699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5600f-9645-4eea-9d4c-950ae3f088c7 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:09.223699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5600f-9645-4eea-9d4c-950ae3f088c7 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:09.225228 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 168/335] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:09.230947 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc62be7-dc1c-45e0-ac42-4c7762be6e90 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:09.231390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc62be7-dc1c-45e0-ac42-4c7762be6e90 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:09.231570 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc62be7-dc1c-45e0-ac42-4c7762be6e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:09.255009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-4bc62be7-dc1c-45e0-ac42-4c7762be6e90 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:09.256247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-4bc62be7-dc1c-45e0-ac42-4c7762be6e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:09.349429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 168/336] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:09.508956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcc7d8-36bb-4198-93fc-af23f376fa49 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:09.509249 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcc7d8-36bb-4198-93fc-af23f376fa49 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:09.509486 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcc7d8-36bb-4198-93fc-af23f376fa49 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:09.511597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 169/337] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:09.517182 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c49436-3765-49b5-bc35-121dd669b486 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:09.517468 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c49436-3765-49b5-bc35-121dd669b486 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:09.517741 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c49436-3765-49b5-bc35-121dd669b486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:09.538611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-f3c49436-3765-49b5-bc35-121dd669b486 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:09.539863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f3c49436-3765-49b5-bc35-121dd669b486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:09.631114 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 169/338] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:09.802313 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b454d-3965-43ef-9fef-be9ac1747b61 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:09.802932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b454d-3965-43ef-9fef-be9ac1747b61 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:09.803403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b454d-3965-43ef-9fef-be9ac1747b61 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:09.805209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 170/339] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:09.810397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47f0b6-cd04-47a1-a6f7-68ff37e175f5 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:09.810682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47f0b6-cd04-47a1-a6f7-68ff37e175f5 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:09.810954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47f0b6-cd04-47a1-a6f7-68ff37e175f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:09.836271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-ff47f0b6-cd04-47a1-a6f7-68ff37e175f5 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:09.837478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ff47f0b6-cd04-47a1-a6f7-68ff37e175f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:09.906204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:09.926382 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 170/340] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:10:09.957106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b454d-3965-43ef-9fef-be9ac1747b61 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:10.055152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:10.081938 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b454d-3965-43ef-9fef-be9ac1747b61 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:10.123141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b68ba0-6e7c-4429-bad4-9076dfef62bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:10.124155 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b68ba0-6e7c-4429-bad4-9076dfef62bd None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:10.124633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b68ba0-6e7c-4429-bad4-9076dfef62bd None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:10.125121 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b68ba0-6e7c-4429-bad4-9076dfef62bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:10.136599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b68ba0-6e7c-4429-bad4-9076dfef62bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:10.145758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-19b68ba0-6e7c-4429-bad4-9076dfef62bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:10:10.145758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:10.145758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:10.145758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:10.145758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:10.145758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:10.145758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:10.145758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:10.145758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:10.145758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:10.145758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:10.145758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:10.145758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:10.145758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:10.145758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:10.145758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:10.146640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:10.146640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:10.146640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:10.146640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:10.146640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:10.146640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:10.146640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:10.146640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:10.146640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:10.146640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:10.146640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:10.146640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:10.146640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:10.146640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:10.146640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:10.146640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:10.147432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:10.147432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:10.147432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:10.147432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:10.147432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:10.147432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:10.147432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:10.147432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:10.147432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:10.147432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:10.147432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:10:10.147432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:10.149971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 171/341] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:10.156598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47f0b6-cd04-47a1-a6f7-68ff37e175f5 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:10.198344 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b68ba0-6e7c-4429-bad4-9076dfef62bd None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:10.292016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47f0b6-cd04-47a1-a6f7-68ff37e175f5 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:10.333470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962d7b0-306b-465a-9246-56ed0922f0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:10.334009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962d7b0-306b-465a-9246-56ed0922f0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:10.334286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962d7b0-306b-465a-9246-56ed0922f0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:10.334555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962d7b0-306b-465a-9246-56ed0922f0f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:10.335224 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962d7b0-306b-465a-9246-56ed0922f0f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:10.335526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962d7b0-306b-465a-9246-56ed0922f0f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:10.336933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962d7b0-306b-465a-9246-56ed0922f0f2 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:10.346398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 171/342] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:10:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:10.356801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962d7b0-306b-465a-9246-56ed0922f0f2 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:10.393961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b68ba0-6e7c-4429-bad4-9076dfef62bd None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:10.428170 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:10.468502 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5d9f1-eee4-4e5c-a898-a503ec1a0b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:10.469151 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5d9f1-eee4-4e5c-a898-a503ec1a0b47 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:10.469430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5d9f1-eee4-4e5c-a898-a503ec1a0b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:10.469704 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5d9f1-eee4-4e5c-a898-a503ec1a0b47 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:10.492698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5d9f1-eee4-4e5c-a898-a503ec1a0b47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:10.498109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-41b5d9f1-eee4-4e5c-a898-a503ec1a0b47 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:10:10.498109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:10.498109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:10.498109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:10.498109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:10.498109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:10.498109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:10.498109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:10.498109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:10.498109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:10.498109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:10.498109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:10.498109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:10.498109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:10.498109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:10.498109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:10.498991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:10.498991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:10.498991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:10.498991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:10.498991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:10.498991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:10.498991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:10.498991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:10.498991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:10.498991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:10.498991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:10.498991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:10.498991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:10.498991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:10.498991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:10.498991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:10.499805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:10.499805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:10.499805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:10.499805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:10.499805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:10.499805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:10.499805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:10.499805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:10.499805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:10.499805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:10.499805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:10:10.499805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:10.499805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 172/343] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:10:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:10:10.502212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962d7b0-306b-465a-9246-56ed0922f0f2 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:10.506722 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5d9f1-eee4-4e5c-a898-a503ec1a0b47 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:10.514323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:10.557919 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5cc62-bf76-4ee4-8c36-7d74aff81b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:10.558816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5cc62-bf76-4ee4-8c36-7d74aff81b58 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:10.559306 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5cc62-bf76-4ee4-8c36-7d74aff81b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:10.559798 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5cc62-bf76-4ee4-8c36-7d74aff81b58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:10.661912 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5cc62-bf76-4ee4-8c36-7d74aff81b58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:10.663554 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5cc62-bf76-4ee4-8c36-7d74aff81b58 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:10.679151 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5d9f1-eee4-4e5c-a898-a503ec1a0b47 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:10.717288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:10.722216 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07d6a6-78d7-442a-83eb-0e788d6fc11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:10.723104 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07d6a6-78d7-442a-83eb-0e788d6fc11d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:10.723588 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07d6a6-78d7-442a-83eb-0e788d6fc11d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:10.724166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07d6a6-78d7-442a-83eb-0e788d6fc11d None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:10.725042 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07d6a6-78d7-442a-83eb-0e788d6fc11d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:10.725588 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07d6a6-78d7-442a-83eb-0e788d6fc11d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:10.728315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07d6a6-78d7-442a-83eb-0e788d6fc11d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:10.738685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 173/344] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:10.740325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0625d-a91b-489d-950a-62db6a36e60b None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:10.740836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0625d-a91b-489d-950a-62db6a36e60b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:10.741309 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0625d-a91b-489d-950a-62db6a36e60b None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:10.742724 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 174/345] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:10.743619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0625d-a91b-489d-950a-62db6a36e60b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:10.775093 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 172/346] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:10:10.777025 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae13293-87f4-4efc-b49d-a6a66d9bfb7f None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:10.777313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae13293-87f4-4efc-b49d-a6a66d9bfb7f None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:10.777584 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae13293-87f4-4efc-b49d-a6a66d9bfb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:10.797826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-fae13293-87f4-4efc-b49d-a6a66d9bfb7f None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:10.799170 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-fae13293-87f4-4efc-b49d-a6a66d9bfb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:10.819268 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:10.882644 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 173/347] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:10.883570 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae13293-87f4-4efc-b49d-a6a66d9bfb7f None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:10.887352 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0625d-a91b-489d-950a-62db6a36e60b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:10.968869 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:10.987669 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae13293-87f4-4efc-b49d-a6a66d9bfb7f None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:10.989082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34626f01-f165-4a89-9d32-9b3f2e268343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:10.989727 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34626f01-f165-4a89-9d32-9b3f2e268343 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:10.990004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34626f01-f165-4a89-9d32-9b3f2e268343 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:10.990271 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34626f01-f165-4a89-9d32-9b3f2e268343 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:11.010913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34626f01-f165-4a89-9d32-9b3f2e268343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:11.023974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-34626f01-f165-4a89-9d32-9b3f2e268343 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:10:11.023974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:11.023974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:11.023974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:11.023974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:11.023974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:11.023974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:11.023974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:11.023974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:11.023974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:11.023974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:11.023974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:11.023974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:11.023974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:11.023974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:11.023974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:11.025179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:11.025179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:11.025179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:11.025179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:11.025179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:11.025179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:11.025179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:11.025179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:11.025179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:11.025179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:11.025179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:11.025179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:11.025179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:11.025179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:11.025179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:11.025179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:11.026466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:11.026466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:11.026466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:11.026466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:11.026466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:11.026466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:11.026466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:11.026466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:11.026466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:11.026466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:11.026466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 11:10:11.026466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:11.026466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 175/348] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:11.026466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34626f01-f165-4a89-9d32-9b3f2e268343 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:11.028555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ee32b-a05d-49a3-8b7b-00d2c6d3d8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:11.029328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ee32b-a05d-49a3-8b7b-00d2c6d3d8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:11.030058 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ee32b-a05d-49a3-8b7b-00d2c6d3d8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:11.030494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ee32b-a05d-49a3-8b7b-00d2c6d3d8a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:11.105084 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ee32b-a05d-49a3-8b7b-00d2c6d3d8a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:11.106772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ee32b-a05d-49a3-8b7b-00d2c6d3d8a7 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:11.204204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:11.212911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 174/349] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:10:11.214763 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ee32b-a05d-49a3-8b7b-00d2c6d3d8a7 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:11.219344 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34626f01-f165-4a89-9d32-9b3f2e268343 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:11.253635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:11.270600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377f0e5-92ea-4945-b207-3aadd8cc9d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:11.271463 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377f0e5-92ea-4945-b207-3aadd8cc9d66 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:11.271949 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377f0e5-92ea-4945-b207-3aadd8cc9d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:11.272392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377f0e5-92ea-4945-b207-3aadd8cc9d66 None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:11.273682 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377f0e5-92ea-4945-b207-3aadd8cc9d66 None admin] RBAC: Authorizing `identity:get_project(project_id=52fe574d05694a42951535f0d8667f77)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:11.275412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377f0e5-92ea-4945-b207-3aadd8cc9d66 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:11.277233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 176/350] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:10:11 2022] GET /identity/v3/projects/52fe574d05694a42951535f0d8667f77 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:11.278179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377f0e5-92ea-4945-b207-3aadd8cc9d66 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:11.324929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ee32b-a05d-49a3-8b7b-00d2c6d3d8a7 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:11.371261 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8c73a-7bd1-4730-a25e-3177d9e399c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:11.372134 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8c73a-7bd1-4730-a25e-3177d9e399c2 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:11.372560 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8c73a-7bd1-4730-a25e-3177d9e399c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:11.372994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8c73a-7bd1-4730-a25e-3177d9e399c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:11.374030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8c73a-7bd1-4730-a25e-3177d9e399c2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:11.375475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8c73a-7bd1-4730-a25e-3177d9e399c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:11.377168 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8c73a-7bd1-4730-a25e-3177d9e399c2 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:11.387994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 175/351] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:10:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:11.389019 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8c73a-7bd1-4730-a25e-3177d9e399c2 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:11.434810 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:11.445729 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377f0e5-92ea-4945-b207-3aadd8cc9d66 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:11.487054 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7b6de-5426-4125-826d-850d6c546c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:11.488132 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7b6de-5426-4125-826d-850d6c546c02 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:11.488613 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7b6de-5426-4125-826d-850d6c546c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:11.489098 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7b6de-5426-4125-826d-850d6c546c02 None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:11.490600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7b6de-5426-4125-826d-850d6c546c02 None admin] RBAC: Authorizing `identity:get_project(project_id=52fe574d05694a42951535f0d8667f77)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:11.492425 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7b6de-5426-4125-826d-850d6c546c02 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:11.494990 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 177/352] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:10:11 2022] GET /identity/v3/projects/52fe574d05694a42951535f0d8667f77 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:11.496025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7b6de-5426-4125-826d-850d6c546c02 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:11.497586 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8c73a-7bd1-4730-a25e-3177d9e399c2 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:11.535464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527d81f-443c-4e63-8f4f-505c389982b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:11.536133 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527d81f-443c-4e63-8f4f-505c389982b5 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:11.536411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527d81f-443c-4e63-8f4f-505c389982b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:11.536680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527d81f-443c-4e63-8f4f-505c389982b5 None admin] PATH_INFO: `/v3/groups/e68ae333760c4cffad23ec61ffad415a` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:11.543220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e527d81f-443c-4e63-8f4f-505c389982b5 None admin] RBAC: Authorizing `identity:get_group(group_id=e68ae333760c4cffad23ec61ffad415a)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:11.544935 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e527d81f-443c-4e63-8f4f-505c389982b5 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:11.546753 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 176/353] 10.222.0.81 () {56 vars in 1180 bytes} [Mon Sep 19 11:10:11 2022] GET /identity/v3/groups/e68ae333760c4cffad23ec61ffad415a => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:11.547717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527d81f-443c-4e63-8f4f-505c389982b5 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:11.570498 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:11.617774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7b6de-5426-4125-826d-850d6c546c02 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:11.662735 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527d81f-443c-4e63-8f4f-505c389982b5 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:11.669007 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9a88c-22a2-424b-ae20-9a9ddcd80c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:11.669997 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9a88c-22a2-424b-ae20-9a9ddcd80c82 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:11.670515 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9a88c-22a2-424b-ae20-9a9ddcd80c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:11.671001 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:11.672247 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9a88c-22a2-424b-ae20-9a9ddcd80c82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:11.687043 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9a88c-22a2-424b-ae20-9a9ddcd80c82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:11.694011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-ebc9a88c-22a2-424b-ae20-9a9ddcd80c82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:10:11.694011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:11.694011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:11.694011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:11.694011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:11.694011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:11.694011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:11.694011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:11.694011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:11.694011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:11.694011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:11.694011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:11.694011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:11.694011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:11.694011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:11.694011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:11.694937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:11.694937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:11.694937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:11.694937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:11.694937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:11.694937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:11.694937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:11.694937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:11.694937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:11.694937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:11.694937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:11.694937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:11.694937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:11.694937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:11.694937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:11.694937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:11.695733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:11.695733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:11.695733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:11.695733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:11.695733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:11.695733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:11.695733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:11.695733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:11.695733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:11.695733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:11.695733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:10:11.695733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:11.698980 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 178/354] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:11.700254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9a88c-22a2-424b-ae20-9a9ddcd80c82 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:11.701092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615d99a-2c99-43db-b1ca-3c05a7b3cbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:11.706429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615d99a-2c99-43db-b1ca-3c05a7b3cbd5 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:11.706429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615d99a-2c99-43db-b1ca-3c05a7b3cbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:11.706429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615d99a-2c99-43db-b1ca-3c05a7b3cbd5 None admin] PATH_INFO: `/v3/users/1274b71a5bbe4b2d91f97043bb735e32` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:11.706429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615d99a-2c99-43db-b1ca-3c05a7b3cbd5 None admin] RBAC: Authorizing `identity:get_user(user_id=1274b71a5bbe4b2d91f97043bb735e32)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:11.706429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615d99a-2c99-43db-b1ca-3c05a7b3cbd5 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:11.708040 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 177/355] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:10:11 2022] GET /identity/v3/users/1274b71a5bbe4b2d91f97043bb735e32 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:11.708795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615d99a-2c99-43db-b1ca-3c05a7b3cbd5 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:11.828308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615d99a-2c99-43db-b1ca-3c05a7b3cbd5 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:11.838545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9a88c-22a2-424b-ae20-9a9ddcd80c82 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:11.860871 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:11.871325 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164413ba-efca-42e4-8438-285c87559def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:11.872236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164413ba-efca-42e4-8438-285c87559def None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:11.872515 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164413ba-efca-42e4-8438-285c87559def None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:11.873700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164413ba-efca-42e4-8438-285c87559def None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:11.873700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164413ba-efca-42e4-8438-285c87559def None admin] RBAC: Adding query filter params (group.id=e68ae333760c4cffad23ec61ffad415a, role.id=0c30c75e227c416da6e4814bce700e8d, scope.project.id=52fe574d05694a42951535f0d8667f77) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:11.873886 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164413ba-efca-42e4-8438-285c87559def None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:11.876084 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164413ba-efca-42e4-8438-285c87559def None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:11.880375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff25e3-56ce-4859-ac3c-1c8d9563daca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:11.881018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff25e3-56ce-4859-ac3c-1c8d9563daca None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:11.881299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff25e3-56ce-4859-ac3c-1c8d9563daca None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:11.881576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff25e3-56ce-4859-ac3c-1c8d9563daca None admin] PATH_INFO: `/v3/groups/e68ae333760c4cffad23ec61ffad415a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:11.882527 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 178/356] 10.222.0.81 () {56 vars in 1376 bytes} [Mon Sep 19 11:10:11 2022] GET /identity/v3/role_assignments?group.id=e68ae333760c4cffad23ec61ffad415a&scope.project.id=52fe574d05694a42951535f0d8667f77&role.id=0c30c75e227c416da6e4814bce700e8d => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:11.882720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff25e3-56ce-4859-ac3c-1c8d9563daca None admin] RBAC: Authorizing `identity:get_group(group_id=e68ae333760c4cffad23ec61ffad415a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:11.883754 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164413ba-efca-42e4-8438-285c87559def None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:11.884274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff25e3-56ce-4859-ac3c-1c8d9563daca None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:11.887210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 179/357] 10.222.0.81 () {56 vars in 1180 bytes} [Mon Sep 19 11:10:11 2022] GET /identity/v3/groups/e68ae333760c4cffad23ec61ffad415a => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:11.888050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff25e3-56ce-4859-ac3c-1c8d9563daca None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:11.935433 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:12.017180 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164413ba-efca-42e4-8438-285c87559def None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:12.027074 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff25e3-56ce-4859-ac3c-1c8d9563daca None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:12.067170 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f43b8-b54b-425a-8ba1-c2070b8a3cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:12.067962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f43b8-b54b-425a-8ba1-c2070b8a3cfb None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:12.068331 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f43b8-b54b-425a-8ba1-c2070b8a3cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:12.068678 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f43b8-b54b-425a-8ba1-c2070b8a3cfb None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:12.069431 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f43b8-b54b-425a-8ba1-c2070b8a3cfb None admin] RBAC: Adding query filter params (name=member) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:12.069807 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f43b8-b54b-425a-8ba1-c2070b8a3cfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:12.071337 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f43b8-b54b-425a-8ba1-c2070b8a3cfb None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:12.079728 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c63c0-561e-4c1b-bb76-b08e9f630a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:12.081466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c63c0-561e-4c1b-bb76-b08e9f630a16 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:12.081930 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c63c0-561e-4c1b-bb76-b08e9f630a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:12.083102 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c63c0-561e-4c1b-bb76-b08e9f630a16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:12.084250 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c63c0-561e-4c1b-bb76-b08e9f630a16 None admin] RBAC: Adding query filter params (role.id=967fbd121d5e4d65b135451eaf2c03d8, user.id=1274b71a5bbe4b2d91f97043bb735e32, scope.system=all) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:12.084962 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c63c0-561e-4c1b-bb76-b08e9f630a16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:12.086568 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 179/358] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:10:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:12.086985 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c63c0-561e-4c1b-bb76-b08e9f630a16 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:12.088563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d11121-8b5c-4b1a-8192-6f44806871d7 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:12.088921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d11121-8b5c-4b1a-8192-6f44806871d7 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:12.089263 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d11121-8b5c-4b1a-8192-6f44806871d7 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:12.090776 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 180/359] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:12.095865 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d11121-8b5c-4b1a-8192-6f44806871d7 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:12.104561 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 180/360] 10.222.0.81 () {56 vars in 1308 bytes} [Mon Sep 19 11:10:11 2022] GET /identity/v3/role_assignments?user.id=1274b71a5bbe4b2d91f97043bb735e32&scope.system=all&role.id=967fbd121d5e4d65b135451eaf2c03d8 => generated 510 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:12.105493 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c63c0-561e-4c1b-bb76-b08e9f630a16 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:12.167936 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:12.229122 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:12.232403 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d11121-8b5c-4b1a-8192-6f44806871d7 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:12.237732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c63c0-561e-4c1b-bb76-b08e9f630a16 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:12.274829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e650f-db06-4e5f-a678-aaca82bbf87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:12.276029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e650f-db06-4e5f-a678-aaca82bbf87f None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:12.276309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e650f-db06-4e5f-a678-aaca82bbf87f None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:12.276588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e650f-db06-4e5f-a678-aaca82bbf87f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:12.277407 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e650f-db06-4e5f-a678-aaca82bbf87f None admin] RBAC: Adding query filter params (group.id=e68ae333760c4cffad23ec61ffad415a, role.id=4cc230d31ac94f4c89bf827491c01bd7, scope.project.id=52fe574d05694a42951535f0d8667f77) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:12.277803 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e650f-db06-4e5f-a678-aaca82bbf87f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:12.278648 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a809359-9d48-416f-b9de-c5a01dc2a7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:12.279271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e650f-db06-4e5f-a678-aaca82bbf87f None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:12.279796 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a809359-9d48-416f-b9de-c5a01dc2a7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:12.280266 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a809359-9d48-416f-b9de-c5a01dc2a7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:12.280739 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a809359-9d48-416f-b9de-c5a01dc2a7b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:12.285941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 181/361] 10.222.0.81 () {56 vars in 1376 bytes} [Mon Sep 19 11:10:12 2022] GET /identity/v3/role_assignments?group.id=e68ae333760c4cffad23ec61ffad415a&scope.project.id=52fe574d05694a42951535f0d8667f77&role.id=4cc230d31ac94f4c89bf827491c01bd7 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:12.287191 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639b1eb-befd-4e1c-a6d1-89d786407a2a None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:12.287480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639b1eb-befd-4e1c-a6d1-89d786407a2a None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:12.292527 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639b1eb-befd-4e1c-a6d1-89d786407a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:12.333115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-9639b1eb-befd-4e1c-a6d1-89d786407a2a None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:12.334750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-9639b1eb-befd-4e1c-a6d1-89d786407a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:12.364287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a809359-9d48-416f-b9de-c5a01dc2a7b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:12.366254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a809359-9d48-416f-b9de-c5a01dc2a7b6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:12.424037 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:12.473484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 182/362] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:12.475112 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9bcb0-2010-4d5f-a0ef-b874aa793975 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:12.475733 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9bcb0-2010-4d5f-a0ef-b874aa793975 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:12.476240 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9bcb0-2010-4d5f-a0ef-b874aa793975 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:12.477788 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 183/363] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:12.485303 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed775532-fc6f-4cb5-8800-85a076eac930 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:12.485812 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed775532-fc6f-4cb5-8800-85a076eac930 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:12.486286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed775532-fc6f-4cb5-8800-85a076eac930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:12.512186 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 181/364] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:10:12.516131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-ed775532-fc6f-4cb5-8800-85a076eac930 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:12.517663 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ed775532-fc6f-4cb5-8800-85a076eac930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:12.518168 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a809359-9d48-416f-b9de-c5a01dc2a7b6 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:12.631427 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 184/365] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:12.668492 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a809359-9d48-416f-b9de-c5a01dc2a7b6 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:12.706066 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34d0e6-dc86-4a0a-8f43-6cf977bd5fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:12.706705 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34d0e6-dc86-4a0a-8f43-6cf977bd5fed None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:12.706981 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34d0e6-dc86-4a0a-8f43-6cf977bd5fed None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:12.707253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34d0e6-dc86-4a0a-8f43-6cf977bd5fed None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:12.708408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e34d0e6-dc86-4a0a-8f43-6cf977bd5fed None admin] RBAC: Authorizing `identity:get_project(project_id=8eb05b28107f4365a8f3ff9b192ce72f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:12.709852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e34d0e6-dc86-4a0a-8f43-6cf977bd5fed None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:12.711848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 182/366] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:10:12 2022] GET /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:12.716968 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed775532-fc6f-4cb5-8800-85a076eac930 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:12.749853 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:12.841252 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed775532-fc6f-4cb5-8800-85a076eac930 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:12.878872 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61f4d7-6ef4-40fb-bf1f-efbfb74aab96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:12.879516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61f4d7-6ef4-40fb-bf1f-efbfb74aab96 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:12.879808 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61f4d7-6ef4-40fb-bf1f-efbfb74aab96 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:12.880084 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61f4d7-6ef4-40fb-bf1f-efbfb74aab96 None admin] PATH_INFO: `/v3/groups/e68ae333760c4cffad23ec61ffad415a` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:12.881220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61f4d7-6ef4-40fb-bf1f-efbfb74aab96 None admin] RBAC: Authorizing `identity:get_group(group_id=e68ae333760c4cffad23ec61ffad415a)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:12.882727 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61f4d7-6ef4-40fb-bf1f-efbfb74aab96 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:12.884608 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 185/367] 10.222.0.81 () {56 vars in 1180 bytes} [Mon Sep 19 11:10:12 2022] GET /identity/v3/groups/e68ae333760c4cffad23ec61ffad415a => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:12.890247 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34d0e6-dc86-4a0a-8f43-6cf977bd5fed None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:12.914806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:12.972656 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61f4d7-6ef4-40fb-bf1f-efbfb74aab96 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:13.053546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34d0e6-dc86-4a0a-8f43-6cf977bd5fed None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:13.095162 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:13.102289 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0fa9f-a214-410e-b26b-ad11449765de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:13.102917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0fa9f-a214-410e-b26b-ad11449765de None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:13.103197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0fa9f-a214-410e-b26b-ad11449765de None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:13.103655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0fa9f-a214-410e-b26b-ad11449765de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:13.104572 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0fa9f-a214-410e-b26b-ad11449765de None admin] RBAC: Adding query filter params (group.id=e68ae333760c4cffad23ec61ffad415a, role.id=0c30c75e227c416da6e4814bce700e8d, scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:13.104879 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0fa9f-a214-410e-b26b-ad11449765de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:13.106307 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0fa9f-a214-410e-b26b-ad11449765de None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:13.106558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61f4d7-6ef4-40fb-bf1f-efbfb74aab96 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:13.115684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 183/368] 10.222.0.81 () {56 vars in 1376 bytes} [Mon Sep 19 11:10:12 2022] GET /identity/v3/role_assignments?group.id=e68ae333760c4cffad23ec61ffad415a&scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f&role.id=0c30c75e227c416da6e4814bce700e8d => generated 260 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:13.120626 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0fa9f-a214-410e-b26b-ad11449765de None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:13.204620 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec0513-848a-4a91-91a4-4082226b42b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:13.205195 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec0513-848a-4a91-91a4-4082226b42b9 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:13.205475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec0513-848a-4a91-91a4-4082226b42b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:13.205757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec0513-848a-4a91-91a4-4082226b42b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:13.236809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec0513-848a-4a91-91a4-4082226b42b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:13.243096 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:13.248392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-12ec0513-848a-4a91-91a4-4082226b42b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:10:13.248392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:13.248392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:13.248392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:13.248392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:13.248392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:13.248392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:13.248392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:13.248392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:13.248392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:13.248392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:13.248392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:13.248392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:13.248392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:13.248392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:13.248392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:13.249458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:13.249458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:13.249458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:13.249458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:13.249458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:13.249458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:13.249458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:13.249458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:13.249458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:13.249458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:13.249458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:13.249458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:13.249458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:13.249458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:13.249458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:13.249458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:13.250948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:13.250948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:13.250948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:13.250948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:13.250948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:13.250948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:13.250948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:13.250948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:13.250948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:13.250948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:13.250948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:10:13.250948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:13.250948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 186/369] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:10:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:10:13.256543 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec0513-848a-4a91-91a4-4082226b42b9 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:13.277369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0fa9f-a214-410e-b26b-ad11449765de None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:13.321756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e158064-2835-4a97-9dac-c59a2df38044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:13.323397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e158064-2835-4a97-9dac-c59a2df38044 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:13.323915 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e158064-2835-4a97-9dac-c59a2df38044 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:13.324383 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e158064-2835-4a97-9dac-c59a2df38044 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:13.336488 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e158064-2835-4a97-9dac-c59a2df38044 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:13.342909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-3e158064-2835-4a97-9dac-c59a2df38044 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:10:13.342909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:13.342909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:13.342909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:13.342909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:13.342909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:13.342909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:13.342909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:13.342909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:13.342909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:13.342909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:13.342909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:13.342909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:13.342909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:13.342909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:13.342909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:13.344050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:13.344050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:13.344050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:13.344050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:13.344050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:13.344050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:13.344050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:13.344050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:13.344050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:13.344050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:13.344050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:13.344050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:13.344050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:13.344050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:13.344050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:13.344050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:13.344890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:13.344890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:13.344890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:13.344890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:13.344890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:13.344890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:13.344890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:13.344890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:13.344890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:13.344890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:13.344890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:10:13.344890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:13.347105 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 184/370] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:10:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:13.356930 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e158064-2835-4a97-9dac-c59a2df38044 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:13.475972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec0513-848a-4a91-91a4-4082226b42b9 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:13.530112 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaffc52-deb1-4e5c-ae72-0342b94dcd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:13.531174 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaffc52-deb1-4e5c-ae72-0342b94dcd70 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:13.531750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaffc52-deb1-4e5c-ae72-0342b94dcd70 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:13.532407 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaffc52-deb1-4e5c-ae72-0342b94dcd70 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:13.534382 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaffc52-deb1-4e5c-ae72-0342b94dcd70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:13.535077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaffc52-deb1-4e5c-ae72-0342b94dcd70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:13.537826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaffc52-deb1-4e5c-ae72-0342b94dcd70 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:13.555001 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e158064-2835-4a97-9dac-c59a2df38044 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:13.557734 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 187/371] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:13.564049 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaffc52-deb1-4e5c-ae72-0342b94dcd70 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:13.593320 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:13.629977 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ce3db-f9bd-4c1c-970c-df8b841bec23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:13.635099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ce3db-f9bd-4c1c-970c-df8b841bec23 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:13.635618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ce3db-f9bd-4c1c-970c-df8b841bec23 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:13.636108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ce3db-f9bd-4c1c-970c-df8b841bec23 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:13.636995 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ce3db-f9bd-4c1c-970c-df8b841bec23 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:13.637560 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ce3db-f9bd-4c1c-970c-df8b841bec23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:13.639219 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ce3db-f9bd-4c1c-970c-df8b841bec23 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:13.651055 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:13.657285 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 185/372] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:13.664444 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ce3db-f9bd-4c1c-970c-df8b841bec23 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:13.769045 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaffc52-deb1-4e5c-ae72-0342b94dcd70 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:13.789781 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ce3db-f9bd-4c1c-970c-df8b841bec23 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:13.827797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c8fcf-3389-4ca1-b4b6-8b7adbc72483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:13.828700 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c8fcf-3389-4ca1-b4b6-8b7adbc72483 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:13.829408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c8fcf-3389-4ca1-b4b6-8b7adbc72483 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:13.829681 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c8fcf-3389-4ca1-b4b6-8b7adbc72483 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:13.860488 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e763edf-2f36-413f-8105-35bbbd01f020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:13.861104 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e763edf-2f36-413f-8105-35bbbd01f020 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:13.861382 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e763edf-2f36-413f-8105-35bbbd01f020 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:13.861653 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e763edf-2f36-413f-8105-35bbbd01f020 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:13.905437 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:13.908610 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c8fcf-3389-4ca1-b4b6-8b7adbc72483 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:13.910355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c8fcf-3389-4ca1-b4b6-8b7adbc72483 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:14.004165 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e763edf-2f36-413f-8105-35bbbd01f020 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:14.005659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e763edf-2f36-413f-8105-35bbbd01f020 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:14.028861 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:14.036672 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 186/373] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:10:14.042996 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c8fcf-3389-4ca1-b4b6-8b7adbc72483 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:14.141017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 188/374] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:10:14.146696 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e763edf-2f36-413f-8105-35bbbd01f020 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:14.171153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c8fcf-3389-4ca1-b4b6-8b7adbc72483 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:14.184126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:14.250405 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ad9f1-ec2c-44d3-98e3-948bd3235349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:14.251023 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ad9f1-ec2c-44d3-98e3-948bd3235349 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:14.251370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ad9f1-ec2c-44d3-98e3-948bd3235349 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:14.251652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ad9f1-ec2c-44d3-98e3-948bd3235349 None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:14.252969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ad9f1-ec2c-44d3-98e3-948bd3235349 None admin] RBAC: Authorizing `identity:get_project(project_id=8eb05b28107f4365a8f3ff9b192ce72f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:14.254600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ad9f1-ec2c-44d3-98e3-948bd3235349 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:14.261334 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 187/375] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:10:14 2022] GET /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:14.267163 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ad9f1-ec2c-44d3-98e3-948bd3235349 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:14.304772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e763edf-2f36-413f-8105-35bbbd01f020 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:14.322475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:14.355002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cddd14-6766-4a04-87ae-830a9efe5934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:14.355643 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cddd14-6766-4a04-87ae-830a9efe5934 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:14.355946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cddd14-6766-4a04-87ae-830a9efe5934 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:14.356219 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cddd14-6766-4a04-87ae-830a9efe5934 None admin] PATH_INFO: `/v3/projects/5d5c1387f77e4df7ac9b4d83d404bc68` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:14.360271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cddd14-6766-4a04-87ae-830a9efe5934 None admin] RBAC: Authorizing `identity:get_project(project_id=5d5c1387f77e4df7ac9b4d83d404bc68)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:14.361723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cddd14-6766-4a04-87ae-830a9efe5934 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:14.363379 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 189/376] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:10:14 2022] GET /identity/v3/projects/5d5c1387f77e4df7ac9b4d83d404bc68 => generated 344 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:14.368905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cddd14-6766-4a04-87ae-830a9efe5934 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:14.435923 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ad9f1-ec2c-44d3-98e3-948bd3235349 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:14.476812 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b03aa8-50c8-4000-b96c-8c32f1851920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:14.477535 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b03aa8-50c8-4000-b96c-8c32f1851920 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:14.477884 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b03aa8-50c8-4000-b96c-8c32f1851920 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:14.478196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b03aa8-50c8-4000-b96c-8c32f1851920 None admin] PATH_INFO: `/v3/groups/e68ae333760c4cffad23ec61ffad415a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:14.479690 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b03aa8-50c8-4000-b96c-8c32f1851920 None admin] RBAC: Authorizing `identity:get_group(group_id=e68ae333760c4cffad23ec61ffad415a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:14.481590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b03aa8-50c8-4000-b96c-8c32f1851920 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:14.483668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 188/377] 10.222.0.81 () {56 vars in 1180 bytes} [Mon Sep 19 11:10:14 2022] GET /identity/v3/groups/e68ae333760c4cffad23ec61ffad415a => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:14.489798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b03aa8-50c8-4000-b96c-8c32f1851920 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:14.532306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:14.579821 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cddd14-6766-4a04-87ae-830a9efe5934 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:14.609174 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b03aa8-50c8-4000-b96c-8c32f1851920 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:14.649997 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0039ca-037a-4aa2-bfd7-b9c73c91806d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:14.650747 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01f8a8-b41e-4624-8969-4d55f698ecbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:14.651179 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0039ca-037a-4aa2-bfd7-b9c73c91806d None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:14.651468 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0039ca-037a-4aa2-bfd7-b9c73c91806d None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:14.651738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0039ca-037a-4aa2-bfd7-b9c73c91806d None admin] PATH_INFO: `/v3/groups/337ef65b0777469d82e8a49b4a1d7a3b` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:14.656869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01f8a8-b41e-4624-8969-4d55f698ecbe None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:14.657178 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01f8a8-b41e-4624-8969-4d55f698ecbe None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:14.657557 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01f8a8-b41e-4624-8969-4d55f698ecbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:14.658358 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01f8a8-b41e-4624-8969-4d55f698ecbe None admin] RBAC: Adding query filter params (group.id=e68ae333760c4cffad23ec61ffad415a, role.id=4cc230d31ac94f4c89bf827491c01bd7, scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:14.658754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01f8a8-b41e-4624-8969-4d55f698ecbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:14.660298 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01f8a8-b41e-4624-8969-4d55f698ecbe None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:14.664491 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0039ca-037a-4aa2-bfd7-b9c73c91806d None admin] RBAC: Authorizing `identity:get_group(group_id=337ef65b0777469d82e8a49b4a1d7a3b)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:14.666054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0039ca-037a-4aa2-bfd7-b9c73c91806d None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:14.668156 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 190/378] 10.222.0.81 () {56 vars in 1180 bytes} [Mon Sep 19 11:10:14 2022] GET /identity/v3/groups/337ef65b0777469d82e8a49b4a1d7a3b => generated 229 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:14.672133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 189/379] 10.222.0.81 () {56 vars in 1376 bytes} [Mon Sep 19 11:10:14 2022] GET /identity/v3/role_assignments?group.id=e68ae333760c4cffad23ec61ffad415a&scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f&role.id=4cc230d31ac94f4c89bf827491c01bd7 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:14.674160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0039ca-037a-4aa2-bfd7-b9c73c91806d None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:14.755305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:14.778797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7931162-09cd-444e-b92e-9b50f6279d12 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:14.779435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7931162-09cd-444e-b92e-9b50f6279d12 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:14.779958 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7931162-09cd-444e-b92e-9b50f6279d12 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:14.781808 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 190/380] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:14.786797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52767c72-484b-4822-8134-d15845e31ce2 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:14.787430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52767c72-484b-4822-8134-d15845e31ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:14.787928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52767c72-484b-4822-8134-d15845e31ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:14.821270 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-52767c72-484b-4822-8134-d15845e31ce2 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:14.823989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-52767c72-484b-4822-8134-d15845e31ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:14.886206 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0039ca-037a-4aa2-bfd7-b9c73c91806d None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:14.954341 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 191/381] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:10:14.961845 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b5fa1-971e-4aff-82ba-4da0a039c723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:14.962806 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b5fa1-971e-4aff-82ba-4da0a039c723 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:14.963240 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b5fa1-971e-4aff-82ba-4da0a039c723 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:14.963667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b5fa1-971e-4aff-82ba-4da0a039c723 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:14.964641 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b5fa1-971e-4aff-82ba-4da0a039c723 None admin] RBAC: Adding query filter params (group.id=337ef65b0777469d82e8a49b4a1d7a3b, role.id=753f8b35d5f0458d8f62d0ad6e28033f, scope.project.id=5d5c1387f77e4df7ac9b4d83d404bc68) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:14.965199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b5fa1-971e-4aff-82ba-4da0a039c723 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:14.966860 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b5fa1-971e-4aff-82ba-4da0a039c723 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:14.974703 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 191/382] 10.222.0.81 () {56 vars in 1376 bytes} [Mon Sep 19 11:10:14 2022] GET /identity/v3/role_assignments?group.id=337ef65b0777469d82e8a49b4a1d7a3b&scope.project.id=5d5c1387f77e4df7ac9b4d83d404bc68&role.id=753f8b35d5f0458d8f62d0ad6e28033f => generated 260 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:15.172786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d13956-135c-4a19-b71f-a49521a0555c None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:15.173286 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d13956-135c-4a19-b71f-a49521a0555c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:15.173725 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d13956-135c-4a19-b71f-a49521a0555c None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:15.175639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 192/383] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:15.182189 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298e742-6282-4a22-9159-be56fea7ceb5 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:15.182466 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298e742-6282-4a22-9159-be56fea7ceb5 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:15.182743 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298e742-6282-4a22-9159-be56fea7ceb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:15.204953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-6298e742-6282-4a22-9159-be56fea7ceb5 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:15.206158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-6298e742-6282-4a22-9159-be56fea7ceb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:15.263154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d13956-135c-4a19-b71f-a49521a0555c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:15.285994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 192/384] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:15.303389 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:15.388586 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d13956-135c-4a19-b71f-a49521a0555c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:15.426815 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39721267-eaef-46b5-b614-2b4f151064b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:15.427376 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39721267-eaef-46b5-b614-2b4f151064b3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:15.427657 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39721267-eaef-46b5-b614-2b4f151064b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:15.427963 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39721267-eaef-46b5-b614-2b4f151064b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:15.441190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39721267-eaef-46b5-b614-2b4f151064b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:15.446119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-39721267-eaef-46b5-b614-2b4f151064b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:10:15.446119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:15.446119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:15.446119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:15.446119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:15.446119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:15.446119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:15.446119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:15.446119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:15.446119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:15.446119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:15.446119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:15.446119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:15.446119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:15.446119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:15.446119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:15.446994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:15.446994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:15.446994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:15.446994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:15.446994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:15.446994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:15.446994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:15.446994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:15.446994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:15.446994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:15.446994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:15.446994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:15.446994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:15.446994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:15.446994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:15.446994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:15.448786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:15.448786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:15.448786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:15.448786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:15.448786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:15.448786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:15.448786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:15.448786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:15.448786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:15.448786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:15.448786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:10:15.448786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:15.448786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 193/385] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:15.454233 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298e742-6282-4a22-9159-be56fea7ceb5 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:15.503094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:15.570642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298e742-6282-4a22-9159-be56fea7ceb5 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:15.610540 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9946ff6-5fa5-4d58-bfa2-7f027a18d91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:15.611206 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9946ff6-5fa5-4d58-bfa2-7f027a18d91e None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:15.611413 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9946ff6-5fa5-4d58-bfa2-7f027a18d91e None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:15.611683 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9946ff6-5fa5-4d58-bfa2-7f027a18d91e None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:15.612379 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9946ff6-5fa5-4d58-bfa2-7f027a18d91e None admin] RBAC: Adding query filter params (name=member) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:15.612682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9946ff6-5fa5-4d58-bfa2-7f027a18d91e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:15.614053 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9946ff6-5fa5-4d58-bfa2-7f027a18d91e None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:15.623362 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 193/386] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:10:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:15.628785 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39721267-eaef-46b5-b614-2b4f151064b3 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:15.750940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:15.761521 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39721267-eaef-46b5-b614-2b4f151064b3 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:15.811262 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41273d8-848e-4f06-89b7-13499a6fd49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:15.812282 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41273d8-848e-4f06-89b7-13499a6fd49e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:15.812644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41273d8-848e-4f06-89b7-13499a6fd49e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:15.813005 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41273d8-848e-4f06-89b7-13499a6fd49e None admin] PATH_INFO: `/v3/groups/e68ae333760c4cffad23ec61ffad415a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:15.815999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41273d8-848e-4f06-89b7-13499a6fd49e None admin] RBAC: Authorizing `identity:get_group(group_id=e68ae333760c4cffad23ec61ffad415a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:15.817664 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41273d8-848e-4f06-89b7-13499a6fd49e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:15.819708 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 194/387] 10.222.0.81 () {56 vars in 1180 bytes} [Mon Sep 19 11:10:15 2022] GET /identity/v3/groups/e68ae333760c4cffad23ec61ffad415a => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:15.825757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9946ff6-5fa5-4d58-bfa2-7f027a18d91e None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:15.831416 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41273d8-848e-4f06-89b7-13499a6fd49e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:15.949084 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9946ff6-5fa5-4d58-bfa2-7f027a18d91e None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:15.952176 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:15.988807 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fefde3-b30d-4449-aa0e-fa3ed354a789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:15.989549 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fefde3-b30d-4449-aa0e-fa3ed354a789 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:15.989898 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fefde3-b30d-4449-aa0e-fa3ed354a789 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:15.990249 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fefde3-b30d-4449-aa0e-fa3ed354a789 None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:15.991474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fefde3-b30d-4449-aa0e-fa3ed354a789 None admin] RBAC: Authorizing `identity:get_project(project_id=52fe574d05694a42951535f0d8667f77)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:15.993019 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fefde3-b30d-4449-aa0e-fa3ed354a789 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:15.995521 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 194/388] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:10:15 2022] GET /identity/v3/projects/52fe574d05694a42951535f0d8667f77 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:16.002275 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fefde3-b30d-4449-aa0e-fa3ed354a789 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:16.035090 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41273d8-848e-4f06-89b7-13499a6fd49e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:16.108315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143117c-349e-4f69-a663-1ec7bcc62583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:16.109411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143117c-349e-4f69-a663-1ec7bcc62583 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:16.109894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143117c-349e-4f69-a663-1ec7bcc62583 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:16.110306 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fefde3-b30d-4449-aa0e-fa3ed354a789 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:16.110591 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143117c-349e-4f69-a663-1ec7bcc62583 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:16.124808 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143117c-349e-4f69-a663-1ec7bcc62583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:16.129660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-1143117c-349e-4f69-a663-1ec7bcc62583 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:10:16.129660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:16.129660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:16.129660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:16.129660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:16.129660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:16.129660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:16.129660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:16.129660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:16.129660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:16.129660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:16.129660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:16.129660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:16.129660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:16.129660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:16.129660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:16.130553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:16.130553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:16.130553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:16.130553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:16.130553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:16.130553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:16.130553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:16.130553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:16.130553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:16.130553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:16.130553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:16.130553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:16.130553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:16.130553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:16.130553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:16.130553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:16.131462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:16.131462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:16.131462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:16.131462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:16.131462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:16.131462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:16.131462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:16.131462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:16.131462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:16.131462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:16.131462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:10:16.131462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:16.131462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 195/389] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:10:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:10:16.143793 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143117c-349e-4f69-a663-1ec7bcc62583 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:16.147724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90dca2b-25f5-40c5-8d7e-ea963f20a269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:16.148763 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dca2b-25f5-40c5-8d7e-ea963f20a269 None admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:16.149052 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dca2b-25f5-40c5-8d7e-ea963f20a269 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:16.149326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dca2b-25f5-40c5-8d7e-ea963f20a269 None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77/groups/e68ae333760c4cffad23ec61ffad415a/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:16.152369 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90dca2b-25f5-40c5-8d7e-ea963f20a269 None admin] RBAC: Authorizing `identity:create_grant(project_id=52fe574d05694a42951535f0d8667f77, group_id=e68ae333760c4cffad23ec61ffad415a, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:16.153829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90dca2b-25f5-40c5-8d7e-ea963f20a269 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:16.173327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:16.176053 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 195/390] 10.222.0.81 () {58 vars in 1442 bytes} [Mon Sep 19 11:10:16 2022] PUT /identity/v3/projects/52fe574d05694a42951535f0d8667f77/groups/e68ae333760c4cffad23ec61ffad415a/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:10:16.177387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e23dfc-a2de-486c-a9f7-e62e22f06996 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:16.178157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e23dfc-a2de-486c-a9f7-e62e22f06996 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:16.178439 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e23dfc-a2de-486c-a9f7-e62e22f06996 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:16.184617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 196/391] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:16.187058 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b49699-3b84-46c4-8a38-5aac374d3ec0 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:16.187344 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b49699-3b84-46c4-8a38-5aac374d3ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:16.187791 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b49699-3b84-46c4-8a38-5aac374d3ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:16.221091 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-e0b49699-3b84-46c4-8a38-5aac374d3ec0 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:16.222636 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e0b49699-3b84-46c4-8a38-5aac374d3ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:16.260496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:16.282467 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143117c-349e-4f69-a663-1ec7bcc62583 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:16.306630 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 197/392] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:10:16.344959 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e561893-7034-4048-a94c-9aa25801c0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:16.345579 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e561893-7034-4048-a94c-9aa25801c0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:16.345916 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e561893-7034-4048-a94c-9aa25801c0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:16.346189 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e561893-7034-4048-a94c-9aa25801c0a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:16.346912 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e561893-7034-4048-a94c-9aa25801c0a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:16.347223 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e561893-7034-4048-a94c-9aa25801c0a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:16.348651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e561893-7034-4048-a94c-9aa25801c0a5 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:16.359848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 196/393] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:16.365557 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b49699-3b84-46c4-8a38-5aac374d3ec0 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:16.492621 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b49699-3b84-46c4-8a38-5aac374d3ec0 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:16.530107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c98e2-d708-4368-9b17-e30c9d7bcfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:16.530669 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c98e2-d708-4368-9b17-e30c9d7bcfcf None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:16.530945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c98e2-d708-4368-9b17-e30c9d7bcfcf None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:16.531223 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c98e2-d708-4368-9b17-e30c9d7bcfcf None admin] PATH_INFO: `/v3/groups/e68ae333760c4cffad23ec61ffad415a` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:16.532367 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c98e2-d708-4368-9b17-e30c9d7bcfcf None admin] RBAC: Authorizing `identity:get_group(group_id=e68ae333760c4cffad23ec61ffad415a)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:16.533796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c98e2-d708-4368-9b17-e30c9d7bcfcf None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:16.535717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 198/394] 10.222.0.81 () {56 vars in 1180 bytes} [Mon Sep 19 11:10:16 2022] GET /identity/v3/groups/e68ae333760c4cffad23ec61ffad415a => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:16.541216 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e561893-7034-4048-a94c-9aa25801c0a5 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:16.628982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:16.649199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e561893-7034-4048-a94c-9aa25801c0a5 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:16.685197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2dff5-8c98-4a3f-92ab-0733ffd52f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:16.685830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2dff5-8c98-4a3f-92ab-0733ffd52f3d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:16.686110 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2dff5-8c98-4a3f-92ab-0733ffd52f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:16.686383 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2dff5-8c98-4a3f-92ab-0733ffd52f3d None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:16.687507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2dff5-8c98-4a3f-92ab-0733ffd52f3d None admin] RBAC: Authorizing `identity:get_project(project_id=52fe574d05694a42951535f0d8667f77)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:16.688969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2dff5-8c98-4a3f-92ab-0733ffd52f3d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:16.690999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 197/395] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:10:16 2022] GET /identity/v3/projects/52fe574d05694a42951535f0d8667f77 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:16.696901 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c98e2-d708-4368-9b17-e30c9d7bcfcf None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:16.726590 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:16.789244 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2dff5-8c98-4a3f-92ab-0733ffd52f3d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:16.805316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c98e2-d708-4368-9b17-e30c9d7bcfcf None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:16.847886 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cd9ee-4d88-48f1-91b8-c56f9ff190f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:16.848560 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cd9ee-4d88-48f1-91b8-c56f9ff190f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:16.848840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cd9ee-4d88-48f1-91b8-c56f9ff190f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:16.849116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cd9ee-4d88-48f1-91b8-c56f9ff190f3 None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77/groups/e68ae333760c4cffad23ec61ffad415a/roles/4cc230d31ac94f4c89bf827491c01bd7` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:16.851569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045cd9ee-4d88-48f1-91b8-c56f9ff190f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=52fe574d05694a42951535f0d8667f77, group_id=e68ae333760c4cffad23ec61ffad415a, role_id=4cc230d31ac94f4c89bf827491c01bd7)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:16.853423 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045cd9ee-4d88-48f1-91b8-c56f9ff190f3 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:16.871369 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 199/396] 10.222.0.81 () {58 vars in 1442 bytes} [Mon Sep 19 11:10:16 2022] PUT /identity/v3/projects/52fe574d05694a42951535f0d8667f77/groups/e68ae333760c4cffad23ec61ffad415a/roles/4cc230d31ac94f4c89bf827491c01bd7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:10:16.982874 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2dff5-8c98-4a3f-92ab-0733ffd52f3d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:17.026306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1836f-7f7b-4613-8cac-9dd18395d452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:17.027347 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1836f-7f7b-4613-8cac-9dd18395d452 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:17.027887 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1836f-7f7b-4613-8cac-9dd18395d452 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:17.028382 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1836f-7f7b-4613-8cac-9dd18395d452 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:17.046217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1836f-7f7b-4613-8cac-9dd18395d452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:17.052092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-e3e1836f-7f7b-4613-8cac-9dd18395d452 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:10:17.052092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:17.052092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:17.052092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:17.052092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:17.052092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:17.052092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:17.052092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:17.052092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:17.052092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:17.052092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:17.052092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:17.052092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:17.052092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:17.052092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:17.052092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:17.053036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:17.053036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:17.053036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:17.053036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:17.053036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:17.053036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:17.053036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:17.053036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:17.053036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:17.053036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:17.053036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:17.053036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:17.053036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:17.053036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:17.053036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:17.053036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:17.053835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:17.053835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:17.053835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:17.053835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:17.053835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:17.053835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:17.053835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:17.053835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:17.053835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:17.053835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:17.053835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:10:17.053835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:17.056773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 198/397] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:10:17.062507 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cd9ee-4d88-48f1-91b8-c56f9ff190f3 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:17.185247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:17.199756 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cd9ee-4d88-48f1-91b8-c56f9ff190f3 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:17.238036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce4846-95dc-44a1-88e2-39e1f1651ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:17.238631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce4846-95dc-44a1-88e2-39e1f1651ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:17.238913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce4846-95dc-44a1-88e2-39e1f1651ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:17.239184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce4846-95dc-44a1-88e2-39e1f1651ae4 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:17.239899 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce4846-95dc-44a1-88e2-39e1f1651ae4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:17.240202 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce4846-95dc-44a1-88e2-39e1f1651ae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:17.241584 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce4846-95dc-44a1-88e2-39e1f1651ae4 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:17.251647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 200/398] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:10:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:17.256965 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1836f-7f7b-4613-8cac-9dd18395d452 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:17.274917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:17.388400 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1836f-7f7b-4613-8cac-9dd18395d452 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:17.428631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a86e6-3492-4737-b2ca-10956301ab7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:17.429788 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a86e6-3492-4737-b2ca-10956301ab7f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:17.430273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a86e6-3492-4737-b2ca-10956301ab7f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:17.430743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a86e6-3492-4737-b2ca-10956301ab7f None admin] PATH_INFO: `/v3/groups/e68ae333760c4cffad23ec61ffad415a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:17.432641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a86e6-3492-4737-b2ca-10956301ab7f None admin] RBAC: Authorizing `identity:get_group(group_id=e68ae333760c4cffad23ec61ffad415a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:17.434291 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a86e6-3492-4737-b2ca-10956301ab7f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:17.436468 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 199/399] 10.222.0.81 () {56 vars in 1180 bytes} [Mon Sep 19 11:10:17 2022] GET /identity/v3/groups/e68ae333760c4cffad23ec61ffad415a => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:17.441712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce4846-95dc-44a1-88e2-39e1f1651ae4 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:17.586672 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce4846-95dc-44a1-88e2-39e1f1651ae4 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:17.631471 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c0c8c-7b79-4913-abe3-d36ad793b2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:17.632130 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c0c8c-7b79-4913-abe3-d36ad793b2ed None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:17.632407 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c0c8c-7b79-4913-abe3-d36ad793b2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:17.632673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c0c8c-7b79-4913-abe3-d36ad793b2ed None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:17.633815 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c0c8c-7b79-4913-abe3-d36ad793b2ed None admin] RBAC: Authorizing `identity:get_project(project_id=8eb05b28107f4365a8f3ff9b192ce72f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:17.635310 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c0c8c-7b79-4913-abe3-d36ad793b2ed None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:17.636967 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 201/400] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:10:17 2022] GET /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:17.643005 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a86e6-3492-4737-b2ca-10956301ab7f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:17.723811 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0962538-29c5-4360-a73d-c0a2df03271f None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:17.723811 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0962538-29c5-4360-a73d-c0a2df03271f None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:17.723811 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0962538-29c5-4360-a73d-c0a2df03271f None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:17.728051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 202/401] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:17.730152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a00b9-dbc4-489c-b8e7-7833ef096c02 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:17.730588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a00b9-dbc4-489c-b8e7-7833ef096c02 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:17.731030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a00b9-dbc4-489c-b8e7-7833ef096c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:17.776765 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-758a00b9-dbc4-489c-b8e7-7833ef096c02 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:17.778195 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-758a00b9-dbc4-489c-b8e7-7833ef096c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:17.786851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:17.816131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:17.820708 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a86e6-3492-4737-b2ca-10956301ab7f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:17.870884 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca429d-3b7a-4a73-b558-2ee8fd4ec0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:17.871783 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca429d-3b7a-4a73-b558-2ee8fd4ec0a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:17.872164 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca429d-3b7a-4a73-b558-2ee8fd4ec0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:17.872538 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca429d-3b7a-4a73-b558-2ee8fd4ec0a9 None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f/groups/e68ae333760c4cffad23ec61ffad415a/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:17.874583 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca429d-3b7a-4a73-b558-2ee8fd4ec0a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=8eb05b28107f4365a8f3ff9b192ce72f, group_id=e68ae333760c4cffad23ec61ffad415a, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:17.876258 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca429d-3b7a-4a73-b558-2ee8fd4ec0a9 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:17.899814 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 200/402] 10.222.0.81 () {58 vars in 1442 bytes} [Mon Sep 19 11:10:17 2022] PUT /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f/groups/e68ae333760c4cffad23ec61ffad415a/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:10:17.899814 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71097362-e117-4211-a941-d842d1644762 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:17.899814 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71097362-e117-4211-a941-d842d1644762 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:17.899814 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71097362-e117-4211-a941-d842d1644762 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:17.899814 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 201/403] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:17.906058 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eca3cd-cf82-47e2-8b38-0ee49b81b088 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:17.906521 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eca3cd-cf82-47e2-8b38-0ee49b81b088 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:17.906899 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eca3cd-cf82-47e2-8b38-0ee49b81b088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:17.924850 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 203/404] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:10:17.930614 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-19eca3cd-cf82-47e2-8b38-0ee49b81b088 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:17.932748 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-19eca3cd-cf82-47e2-8b38-0ee49b81b088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:18.013069 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 202/405] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:18.304886 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a00b9-dbc4-489c-b8e7-7833ef096c02 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:18.419001 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eca3cd-cf82-47e2-8b38-0ee49b81b088 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:18.428642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a00b9-dbc4-489c-b8e7-7833ef096c02 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:18.478869 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30db44a-e1c7-489b-8fb9-6fbe7985272f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:18.479871 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30db44a-e1c7-489b-8fb9-6fbe7985272f None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:18.480288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30db44a-e1c7-489b-8fb9-6fbe7985272f None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:18.480722 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30db44a-e1c7-489b-8fb9-6fbe7985272f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:18.492019 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30db44a-e1c7-489b-8fb9-6fbe7985272f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:18.497532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-b30db44a-e1c7-489b-8fb9-6fbe7985272f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:10:18.497532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:18.497532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:18.497532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:18.497532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:18.497532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:18.497532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:18.497532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:18.497532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:18.497532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:18.497532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:18.497532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:18.497532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:18.497532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:18.497532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:18.497532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:18.498617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:18.498617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:18.498617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:18.498617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:18.498617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:18.498617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:18.498617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:18.498617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:18.498617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:18.498617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:18.498617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:18.498617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:18.498617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:18.498617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:18.498617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:18.498617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:18.499581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:18.499581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:18.499581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:18.499581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:18.499581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:18.499581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:18.499581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:18.499581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:18.499581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:18.499581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:18.499581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:10:18.499581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:18.501900 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 204/406] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:10:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:18.508959 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30db44a-e1c7-489b-8fb9-6fbe7985272f None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:18.516558 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:18.536410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:18.571301 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eca3cd-cf82-47e2-8b38-0ee49b81b088 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:18.609790 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92164032-cdb1-4be4-95f6-b2068aa9ff89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:18.610673 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92164032-cdb1-4be4-95f6-b2068aa9ff89 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:18.610956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92164032-cdb1-4be4-95f6-b2068aa9ff89 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:18.611231 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92164032-cdb1-4be4-95f6-b2068aa9ff89 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:18.623774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92164032-cdb1-4be4-95f6-b2068aa9ff89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:18.629413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-92164032-cdb1-4be4-95f6-b2068aa9ff89 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:10:18.629413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:18.629413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:18.629413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:18.629413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:18.629413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:18.629413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:18.629413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:18.629413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:18.629413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:18.629413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:18.629413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:18.629413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:18.629413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:18.629413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:18.629413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:18.630323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:18.630323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:18.630323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:18.630323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:18.630323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:18.630323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:18.630323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:18.630323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:18.630323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:18.630323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:18.630323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:18.630323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:18.630323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:18.630323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:18.630323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:18.630323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:18.631126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:18.631126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:18.631126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:18.631126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:18.631126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:18.631126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:18.631126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:18.631126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:18.631126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:18.631126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:18.631126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:10:18.631126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:18.632296 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 203/407] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:10:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:18.639297 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92164032-cdb1-4be4-95f6-b2068aa9ff89 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:18.770496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30db44a-e1c7-489b-8fb9-6fbe7985272f None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:18.789000 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92164032-cdb1-4be4-95f6-b2068aa9ff89 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:18.825202 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc4b46-d14e-4559-a10a-af93c72f8e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:18.825719 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc4b46-d14e-4559-a10a-af93c72f8e20 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:18.825997 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc4b46-d14e-4559-a10a-af93c72f8e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:18.826270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc4b46-d14e-4559-a10a-af93c72f8e20 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:18.826938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc4b46-d14e-4559-a10a-af93c72f8e20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:18.827245 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc4b46-d14e-4559-a10a-af93c72f8e20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:18.828897 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc4b46-d14e-4559-a10a-af93c72f8e20 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:18.840308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:18.848971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc6105-b753-4a26-9990-9f0407fd607b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:18.849668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc6105-b753-4a26-9990-9f0407fd607b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:18.849958 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc6105-b753-4a26-9990-9f0407fd607b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:18.850367 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc6105-b753-4a26-9990-9f0407fd607b None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:18.851081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc6105-b753-4a26-9990-9f0407fd607b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:18.851396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc6105-b753-4a26-9990-9f0407fd607b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:18.853059 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc6105-b753-4a26-9990-9f0407fd607b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:18.854808 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 205/408] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:18.862358 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc4b46-d14e-4559-a10a-af93c72f8e20 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:18.869216 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 204/409] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:18.875243 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc6105-b753-4a26-9990-9f0407fd607b None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:19.000259 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc6105-b753-4a26-9990-9f0407fd607b None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:19.013027 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:19.046666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc4b46-d14e-4559-a10a-af93c72f8e20 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:19.046879 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42210ee5-cccc-41d1-94c0-f8f90dc79e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:19.047512 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42210ee5-cccc-41d1-94c0-f8f90dc79e5f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:19.047824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42210ee5-cccc-41d1-94c0-f8f90dc79e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:19.048109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42210ee5-cccc-41d1-94c0-f8f90dc79e5f None admin] PATH_INFO: `/v3/groups/e68ae333760c4cffad23ec61ffad415a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:19.050415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42210ee5-cccc-41d1-94c0-f8f90dc79e5f None admin] RBAC: Authorizing `identity:get_group(group_id=e68ae333760c4cffad23ec61ffad415a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:19.052061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42210ee5-cccc-41d1-94c0-f8f90dc79e5f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:19.053954 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 205/410] 10.222.0.81 () {56 vars in 1180 bytes} [Mon Sep 19 11:10:18 2022] GET /identity/v3/groups/e68ae333760c4cffad23ec61ffad415a => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:19.057215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52933d1-79e9-4e96-bb06-4515311787a9 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:19.057612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52933d1-79e9-4e96-bb06-4515311787a9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:19.057955 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52933d1-79e9-4e96-bb06-4515311787a9 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:19.063616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 206/411] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:19.064497 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52933d1-79e9-4e96-bb06-4515311787a9 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:19.094007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33c369-a118-4f8f-b10c-43c59a46b231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:19.094727 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33c369-a118-4f8f-b10c-43c59a46b231 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:19.095318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33c369-a118-4f8f-b10c-43c59a46b231 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:19.095612 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33c369-a118-4f8f-b10c-43c59a46b231 None admin] PATH_INFO: `/v3/groups/337ef65b0777469d82e8a49b4a1d7a3b` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:19.097083 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a33c369-a118-4f8f-b10c-43c59a46b231 None admin] RBAC: Authorizing `identity:get_group(group_id=337ef65b0777469d82e8a49b4a1d7a3b)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:19.099274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a33c369-a118-4f8f-b10c-43c59a46b231 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:19.103861 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 206/412] 10.222.0.81 () {56 vars in 1180 bytes} [Mon Sep 19 11:10:18 2022] GET /identity/v3/groups/337ef65b0777469d82e8a49b4a1d7a3b => generated 229 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:19.105217 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460a35b-804a-443e-ae32-36e7b744f9dc None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:19.105565 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460a35b-804a-443e-ae32-36e7b744f9dc None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:19.106068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460a35b-804a-443e-ae32-36e7b744f9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:19.127185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-e460a35b-804a-443e-ae32-36e7b744f9dc None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:19.128402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e460a35b-804a-443e-ae32-36e7b744f9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:19.176185 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:19.180363 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52933d1-79e9-4e96-bb06-4515311787a9 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:19.215944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700245dc-b2f5-46ee-9f83-38628c456008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:19.216571 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700245dc-b2f5-46ee-9f83-38628c456008 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:19.216910 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700245dc-b2f5-46ee-9f83-38628c456008 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:19.217189 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700245dc-b2f5-46ee-9f83-38628c456008 None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:19.218362 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 207/413] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:19.218609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700245dc-b2f5-46ee-9f83-38628c456008 None admin] RBAC: Authorizing `identity:get_project(project_id=8eb05b28107f4365a8f3ff9b192ce72f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:19.219218 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460a35b-804a-443e-ae32-36e7b744f9dc None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:19.220099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700245dc-b2f5-46ee-9f83-38628c456008 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:19.237666 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 207/414] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:10:19 2022] GET /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:19.244593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700245dc-b2f5-46ee-9f83-38628c456008 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:19.304451 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:19.346100 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460a35b-804a-443e-ae32-36e7b744f9dc None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:19.385862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ac3c2-86f0-4acc-8f09-6babdce23319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:19.386506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ac3c2-86f0-4acc-8f09-6babdce23319 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:19.386786 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ac3c2-86f0-4acc-8f09-6babdce23319 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:19.387061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ac3c2-86f0-4acc-8f09-6babdce23319 None admin] PATH_INFO: `/v3/projects/5d5c1387f77e4df7ac9b4d83d404bc68` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:19.388500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ac3c2-86f0-4acc-8f09-6babdce23319 None admin] RBAC: Authorizing `identity:get_project(project_id=5d5c1387f77e4df7ac9b4d83d404bc68)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:19.390150 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ac3c2-86f0-4acc-8f09-6babdce23319 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:19.392239 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 208/415] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:10:19 2022] GET /identity/v3/projects/5d5c1387f77e4df7ac9b4d83d404bc68 => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:19.398506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ac3c2-86f0-4acc-8f09-6babdce23319 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:19.468795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700245dc-b2f5-46ee-9f83-38628c456008 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:19.544119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:19.546760 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ac3c2-86f0-4acc-8f09-6babdce23319 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:19.552776 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81544e-9ad6-451a-a98f-11ab2838defd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:19.553507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81544e-9ad6-451a-a98f-11ab2838defd None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:19.553850 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81544e-9ad6-451a-a98f-11ab2838defd None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:19.554186 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81544e-9ad6-451a-a98f-11ab2838defd None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f/groups/e68ae333760c4cffad23ec61ffad415a/roles/4cc230d31ac94f4c89bf827491c01bd7` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:19.556993 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81544e-9ad6-451a-a98f-11ab2838defd None admin] RBAC: Authorizing `identity:create_grant(project_id=8eb05b28107f4365a8f3ff9b192ce72f, group_id=e68ae333760c4cffad23ec61ffad415a, role_id=4cc230d31ac94f4c89bf827491c01bd7)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:19.558977 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81544e-9ad6-451a-a98f-11ab2838defd None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:19.578107 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 208/416] 10.222.0.81 () {58 vars in 1442 bytes} [Mon Sep 19 11:10:19 2022] PUT /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f/groups/e68ae333760c4cffad23ec61ffad415a/roles/4cc230d31ac94f4c89bf827491c01bd7 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:10:19.579502 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81544e-9ad6-451a-a98f-11ab2838defd None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:19.607254 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241824e-c428-43b4-92d8-2f398419f6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:19.608151 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241824e-c428-43b4-92d8-2f398419f6a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:19.608431 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241824e-c428-43b4-92d8-2f398419f6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:19.608702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241824e-c428-43b4-92d8-2f398419f6a3 None admin] PATH_INFO: `/v3/projects/5d5c1387f77e4df7ac9b4d83d404bc68/groups/337ef65b0777469d82e8a49b4a1d7a3b/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:19.610826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241824e-c428-43b4-92d8-2f398419f6a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d5c1387f77e4df7ac9b4d83d404bc68, group_id=337ef65b0777469d82e8a49b4a1d7a3b, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:19.612351 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241824e-c428-43b4-92d8-2f398419f6a3 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:19.616001 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:19.639383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 209/417] 10.222.0.81 () {58 vars in 1442 bytes} [Mon Sep 19 11:10:19 2022] PUT /identity/v3/projects/5d5c1387f77e4df7ac9b4d83d404bc68/groups/337ef65b0777469d82e8a49b4a1d7a3b/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:10:19.780120 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521843a-9fc6-48f5-8ecb-15518f33497e None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:19.780770 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521843a-9fc6-48f5-8ecb-15518f33497e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:19.780770 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521843a-9fc6-48f5-8ecb-15518f33497e None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:19.782484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 210/418] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:10:19.788575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ea61c-4454-4cc8-b0c5-d2fbc8dfd8fd None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:19.788787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81544e-9ad6-451a-a98f-11ab2838defd None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:19.788878 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ea61c-4454-4cc8-b0c5-d2fbc8dfd8fd None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:19.789208 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ea61c-4454-4cc8-b0c5-d2fbc8dfd8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:19.820118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-3c8ea61c-4454-4cc8-b0c5-d2fbc8dfd8fd None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:19.821354 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3c8ea61c-4454-4cc8-b0c5-d2fbc8dfd8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:19.852811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208830b-0b71-4454-99b2-a6d0122a4eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:19.854086 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208830b-0b71-4454-99b2-a6d0122a4eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:19.854304 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208830b-0b71-4454-99b2-a6d0122a4eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:19.854571 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208830b-0b71-4454-99b2-a6d0122a4eb4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:19.871608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208830b-0b71-4454-99b2-a6d0122a4eb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:19.877628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-c208830b-0b71-4454-99b2-a6d0122a4eb4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:10:19.877628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:19.877628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:19.877628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:19.877628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:19.877628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:19.877628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:19.877628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:19.877628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:19.877628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:19.877628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:19.877628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:19.877628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:19.877628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:19.877628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:19.877628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:19.878540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:19.878540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:19.878540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:19.878540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:19.878540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:19.878540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:19.878540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:19.878540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:19.878540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:19.878540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:19.878540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:19.878540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:19.878540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:19.878540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:19.878540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:19.878540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:19.879353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:19.879353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:19.879353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:19.879353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:19.879353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:19.879353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:19.879353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:19.879353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:19.879353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:19.879353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:19.879353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:10:19.879353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:19.879353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 209/419] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:19.885613 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208830b-0b71-4454-99b2-a6d0122a4eb4 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:19.926107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 211/420] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:10:19.944661 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:20.022398 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208830b-0b71-4454-99b2-a6d0122a4eb4 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:20.072901 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf108017-533c-4b5d-b291-77c1a3c749f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:20.073498 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf108017-533c-4b5d-b291-77c1a3c749f7 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:20.073768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf108017-533c-4b5d-b291-77c1a3c749f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:20.074035 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf108017-533c-4b5d-b291-77c1a3c749f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:20.074707 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf108017-533c-4b5d-b291-77c1a3c749f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:20.075006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf108017-533c-4b5d-b291-77c1a3c749f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:20.076403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf108017-533c-4b5d-b291-77c1a3c749f7 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:20.090812 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 210/421] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:10:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:20.097524 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ea61c-4454-4cc8-b0c5-d2fbc8dfd8fd None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:20.221265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf108017-533c-4b5d-b291-77c1a3c749f7 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:20.333860 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ea61c-4454-4cc8-b0c5-d2fbc8dfd8fd None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:20.374157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:20.386430 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2914e-f91f-4246-8a65-ecb7a2e9cbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:20.387051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2914e-f91f-4246-8a65-ecb7a2e9cbf6 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:20.387326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2914e-f91f-4246-8a65-ecb7a2e9cbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:20.387592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2914e-f91f-4246-8a65-ecb7a2e9cbf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:20.415115 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf108017-533c-4b5d-b291-77c1a3c749f7 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:20.432380 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:20.482045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393d04e-c70a-42f8-9e96-28df0b5c0965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:20.482582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393d04e-c70a-42f8-9e96-28df0b5c0965 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:20.482857 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393d04e-c70a-42f8-9e96-28df0b5c0965 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:20.483129 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393d04e-c70a-42f8-9e96-28df0b5c0965 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:20.502242 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393d04e-c70a-42f8-9e96-28df0b5c0965 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:20.507668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-1393d04e-c70a-42f8-9e96-28df0b5c0965 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:10:20.507668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:20.507668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:20.507668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:20.507668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:20.507668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:20.507668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:20.507668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:20.507668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:20.507668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:20.507668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:20.507668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:20.507668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:20.507668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:20.507668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:20.507668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:20.509433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:20.509433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:20.509433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:20.509433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:20.509433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:20.509433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:20.509433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:20.509433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:20.509433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:20.509433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:20.509433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:20.509433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:20.509433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:20.509433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:20.509433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:20.509433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:20.511061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:20.511061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:20.511061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:20.511061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:20.511061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:20.511061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:20.511061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:20.511061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:20.511061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:20.511061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:20.511061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:10:20.511061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:20.514382 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 211/422] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:10:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:10:20.514638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2914e-f91f-4246-8a65-ecb7a2e9cbf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:20.517831 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393d04e-c70a-42f8-9e96-28df0b5c0965 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:20.521298 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2914e-f91f-4246-8a65-ecb7a2e9cbf6 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:20.717627 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393d04e-c70a-42f8-9e96-28df0b5c0965 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:20.744036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 212/423] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 11:10:20.761749 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2914e-f91f-4246-8a65-ecb7a2e9cbf6 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:20.820918 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa153b9d-30fa-4a41-9a45-6a610a1a5690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:20.822496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa153b9d-30fa-4a41-9a45-6a610a1a5690 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:20.822998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa153b9d-30fa-4a41-9a45-6a610a1a5690 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:20.823479 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa153b9d-30fa-4a41-9a45-6a610a1a5690 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:20.824734 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa153b9d-30fa-4a41-9a45-6a610a1a5690 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:20.825279 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa153b9d-30fa-4a41-9a45-6a610a1a5690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:20.827825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa153b9d-30fa-4a41-9a45-6a610a1a5690 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:20.828182 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:20.844977 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 212/424] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:20.848666 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa153b9d-30fa-4a41-9a45-6a610a1a5690 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:20.900152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:20.906041 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2914e-f91f-4246-8a65-ecb7a2e9cbf6 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:20.984475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e92ad-5f6d-4aa9-a24c-a039731271db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:20.985122 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e92ad-5f6d-4aa9-a24c-a039731271db None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:20.985402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e92ad-5f6d-4aa9-a24c-a039731271db None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:20.985673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e92ad-5f6d-4aa9-a24c-a039731271db None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:20.989167 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e92ad-5f6d-4aa9-a24c-a039731271db None admin] RBAC: Authorizing `identity:get_project(project_id=52fe574d05694a42951535f0d8667f77)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:20.990704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e92ad-5f6d-4aa9-a24c-a039731271db None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:20.995876 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 213/425] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:10:20 2022] GET /identity/v3/projects/52fe574d05694a42951535f0d8667f77 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:21.000247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e92ad-5f6d-4aa9-a24c-a039731271db None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:21.039517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa153b9d-30fa-4a41-9a45-6a610a1a5690 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:21.095355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283b4dd-0622-48ef-8db3-7b6ea1d22d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:21.096035 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283b4dd-0622-48ef-8db3-7b6ea1d22d1b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:21.096327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283b4dd-0622-48ef-8db3-7b6ea1d22d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:21.096607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283b4dd-0622-48ef-8db3-7b6ea1d22d1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:21.137992 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:21.146732 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e92ad-5f6d-4aa9-a24c-a039731271db None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:21.177169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283b4dd-0622-48ef-8db3-7b6ea1d22d1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:21.179165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283b4dd-0622-48ef-8db3-7b6ea1d22d1b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:21.187776 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c2a1d-d648-4cc3-820e-9a19bd978bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:21.188565 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c2a1d-d648-4cc3-820e-9a19bd978bba None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:21.188910 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c2a1d-d648-4cc3-820e-9a19bd978bba None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:21.189252 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c2a1d-d648-4cc3-820e-9a19bd978bba None admin] PATH_INFO: `/v3/groups/e68ae333760c4cffad23ec61ffad415a` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:21.190622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c2a1d-d648-4cc3-820e-9a19bd978bba None admin] RBAC: Authorizing `identity:get_group(group_id=e68ae333760c4cffad23ec61ffad415a)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:21.192267 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c2a1d-d648-4cc3-820e-9a19bd978bba None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:21.194519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 214/426] 10.222.0.81 () {56 vars in 1180 bytes} [Mon Sep 19 11:10:20 2022] GET /identity/v3/groups/e68ae333760c4cffad23ec61ffad415a => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:21.196578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5331b87-1ef0-441d-b376-44fb87a3f589 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:21.196927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5331b87-1ef0-441d-b376-44fb87a3f589 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:21.197293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5331b87-1ef0-441d-b376-44fb87a3f589 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:21.202958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 215/427] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:21.204165 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5331b87-1ef0-441d-b376-44fb87a3f589 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:21.208829 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:21.307264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 213/428] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:10:21.307264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fe7a0-4a26-4a6b-8c62-b276ca354e87 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:21.307264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fe7a0-4a26-4a6b-8c62-b276ca354e87 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:21.307264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fe7a0-4a26-4a6b-8c62-b276ca354e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:21.327855 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8f8fe7a0-4a26-4a6b-8c62-b276ca354e87 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:21.329575 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8f8fe7a0-4a26-4a6b-8c62-b276ca354e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:21.339150 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5331b87-1ef0-441d-b376-44fb87a3f589 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:21.379799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b7b1f-c7fd-496b-b0e9-8bf57d417ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:21.380460 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b7b1f-c7fd-496b-b0e9-8bf57d417ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:21.380727 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b7b1f-c7fd-496b-b0e9-8bf57d417ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:21.380996 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b7b1f-c7fd-496b-b0e9-8bf57d417ba3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:21.381716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b7b1f-c7fd-496b-b0e9-8bf57d417ba3 None admin] RBAC: Adding query filter params (group.id=e68ae333760c4cffad23ec61ffad415a, role.id=0c30c75e227c416da6e4814bce700e8d, scope.project.id=52fe574d05694a42951535f0d8667f77) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:21.382082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b7b1f-c7fd-496b-b0e9-8bf57d417ba3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:21.383515 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b7b1f-c7fd-496b-b0e9-8bf57d417ba3 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:21.390992 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 216/429] 10.222.0.81 () {56 vars in 1376 bytes} [Mon Sep 19 11:10:21 2022] GET /identity/v3/role_assignments?group.id=e68ae333760c4cffad23ec61ffad415a&scope.project.id=52fe574d05694a42951535f0d8667f77&role.id=0c30c75e227c416da6e4814bce700e8d => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:21.391951 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b7b1f-c7fd-496b-b0e9-8bf57d417ba3 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:21.419026 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 214/430] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:21.430493 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:21.515908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b7b1f-c7fd-496b-b0e9-8bf57d417ba3 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:21.557119 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685ee72-f251-476c-8fce-222a5793e938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:21.558195 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685ee72-f251-476c-8fce-222a5793e938 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:21.558670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685ee72-f251-476c-8fce-222a5793e938 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:21.559152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685ee72-f251-476c-8fce-222a5793e938 None admin] PATH_INFO: `/v3/projects/52fe574d05694a42951535f0d8667f77` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:21.564477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685ee72-f251-476c-8fce-222a5793e938 None admin] RBAC: Authorizing `identity:get_project(project_id=52fe574d05694a42951535f0d8667f77)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:21.566265 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685ee72-f251-476c-8fce-222a5793e938 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:21.568272 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 217/431] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:10:21 2022] GET /identity/v3/projects/52fe574d05694a42951535f0d8667f77 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:21.574679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fe7a0-4a26-4a6b-8c62-b276ca354e87 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:21.658606 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:21.712172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685ee72-f251-476c-8fce-222a5793e938 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:21.725719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fe7a0-4a26-4a6b-8c62-b276ca354e87 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:21.771262 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d6724-c758-468c-8251-d83117b2ad7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:21.771987 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d6724-c758-468c-8251-d83117b2ad7d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:21.772267 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d6724-c758-468c-8251-d83117b2ad7d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:21.772598 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d6724-c758-468c-8251-d83117b2ad7d None admin] PATH_INFO: `/v3/groups/e68ae333760c4cffad23ec61ffad415a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:21.775874 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d6724-c758-468c-8251-d83117b2ad7d None admin] RBAC: Authorizing `identity:get_group(group_id=e68ae333760c4cffad23ec61ffad415a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:21.776091 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d6724-c758-468c-8251-d83117b2ad7d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:21.779842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 215/432] 10.222.0.81 () {56 vars in 1180 bytes} [Mon Sep 19 11:10:21 2022] GET /identity/v3/groups/e68ae333760c4cffad23ec61ffad415a => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:21.785499 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d6724-c758-468c-8251-d83117b2ad7d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:21.857405 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:21.868342 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685ee72-f251-476c-8fce-222a5793e938 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:21.909369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d6724-c758-468c-8251-d83117b2ad7d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:21.921030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92f7c8-87f1-43c6-8fdc-645521cca9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:21.922303 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92f7c8-87f1-43c6-8fdc-645521cca9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:21.923004 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92f7c8-87f1-43c6-8fdc-645521cca9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:21.923438 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92f7c8-87f1-43c6-8fdc-645521cca9e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:21.938026 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92f7c8-87f1-43c6-8fdc-645521cca9e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:21.939344 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:21.948311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-cb92f7c8-87f1-43c6-8fdc-645521cca9e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:10:21.948311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:21.948311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:21.948311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:21.948311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:21.948311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:21.948311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:21.948311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:21.948311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:21.948311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:21.948311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:21.948311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:21.948311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:21.948311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:21.948311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:21.948311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:21.949247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:21.949247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:21.949247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:21.949247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:21.949247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:21.949247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:21.949247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:21.949247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:21.949247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:21.949247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:21.949247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:21.949247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:21.949247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:21.949247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:21.949247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:21.949247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:21.950044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:21.950044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:21.950044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:21.950044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:21.950044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:21.950044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:21.950044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:21.950044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:21.950044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:21.950044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:21.950044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 11:10:21.950044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:21.952814 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 218/433] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:21.957132 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d6680-833b-43c7-8bf0-db83f9aaab5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:21.957753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d6680-833b-43c7-8bf0-db83f9aaab5f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:21.958031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d6680-833b-43c7-8bf0-db83f9aaab5f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:21.958297 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d6680-833b-43c7-8bf0-db83f9aaab5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:21.959086 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d6680-833b-43c7-8bf0-db83f9aaab5f None admin] RBAC: Adding query filter params (group.id=e68ae333760c4cffad23ec61ffad415a, role.id=4cc230d31ac94f4c89bf827491c01bd7, scope.project.id=52fe574d05694a42951535f0d8667f77) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:21.959393 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d6680-833b-43c7-8bf0-db83f9aaab5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:21.960852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d6680-833b-43c7-8bf0-db83f9aaab5f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:21.967240 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92f7c8-87f1-43c6-8fdc-645521cca9e2 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:21.971831 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 216/434] 10.222.0.81 () {56 vars in 1376 bytes} [Mon Sep 19 11:10:21 2022] GET /identity/v3/role_assignments?group.id=e68ae333760c4cffad23ec61ffad415a&scope.project.id=52fe574d05694a42951535f0d8667f77&role.id=4cc230d31ac94f4c89bf827491c01bd7 => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:22.094937 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92f7c8-87f1-43c6-8fdc-645521cca9e2 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:22.139464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491f80b-b164-4c1a-a1c6-bf426122a984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:22.140062 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491f80b-b164-4c1a-a1c6-bf426122a984 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:22.140348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491f80b-b164-4c1a-a1c6-bf426122a984 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:22.140626 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491f80b-b164-4c1a-a1c6-bf426122a984 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:22.141338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491f80b-b164-4c1a-a1c6-bf426122a984 None admin] RBAC: Adding query filter params (name=member) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:22.141646 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491f80b-b164-4c1a-a1c6-bf426122a984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:22.143111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491f80b-b164-4c1a-a1c6-bf426122a984 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:22.154019 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 219/435] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:10:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:22.159973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d6680-833b-43c7-8bf0-db83f9aaab5f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:22.276400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54603f-b898-456a-af88-613cfa4ba99e None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:22.276400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54603f-b898-456a-af88-613cfa4ba99e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:22.276667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54603f-b898-456a-af88-613cfa4ba99e None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:22.278265 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 220/436] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:10:22.284737 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac042f03-b5f6-48f6-9bd2-dae5ab90bf17 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:22.285165 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac042f03-b5f6-48f6-9bd2-dae5ab90bf17 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:22.285452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac042f03-b5f6-48f6-9bd2-dae5ab90bf17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:22.291624 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d6680-833b-43c7-8bf0-db83f9aaab5f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:22.315236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-ac042f03-b5f6-48f6-9bd2-dae5ab90bf17 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:22.316527 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ac042f03-b5f6-48f6-9bd2-dae5ab90bf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:22.332010 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:22.337273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823f563-b921-4dc5-ad8c-becbe0ba5f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:22.338801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823f563-b921-4dc5-ad8c-becbe0ba5f45 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:22.339309 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823f563-b921-4dc5-ad8c-becbe0ba5f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:22.339859 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823f563-b921-4dc5-ad8c-becbe0ba5f45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:22.416187 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 221/437] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:10:22.436651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:22.449477 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823f563-b921-4dc5-ad8c-becbe0ba5f45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:22.450953 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823f563-b921-4dc5-ad8c-becbe0ba5f45 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:22.472994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e432d89-4771-4653-ab06-ba5fd18cb590 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:22.473288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e432d89-4771-4653-ab06-ba5fd18cb590 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:22.473540 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e432d89-4771-4653-ab06-ba5fd18cb590 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:22.475003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 222/438] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:10:22.480157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a78fe-01d3-4771-a8ba-6f621da79642 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:22.480554 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a78fe-01d3-4771-a8ba-6f621da79642 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:22.480831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a78fe-01d3-4771-a8ba-6f621da79642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:22.500753 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-690a78fe-01d3-4771-a8ba-6f621da79642 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:22.503797 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-690a78fe-01d3-4771-a8ba-6f621da79642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:22.562478 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 217/439] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:10:22.567938 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823f563-b921-4dc5-ad8c-becbe0ba5f45 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:22.601780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 223/440] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:10:22.665602 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:22.678630 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823f563-b921-4dc5-ad8c-becbe0ba5f45 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:22.704794 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a78fe-01d3-4771-a8ba-6f621da79642 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:22.714860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e44f8-902f-4ad9-8b7e-3300d1073b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:22.715483 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e44f8-902f-4ad9-8b7e-3300d1073b08 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:22.715777 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e44f8-902f-4ad9-8b7e-3300d1073b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:22.716058 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e44f8-902f-4ad9-8b7e-3300d1073b08 None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:22.717173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e44f8-902f-4ad9-8b7e-3300d1073b08 None admin] RBAC: Authorizing `identity:get_project(project_id=8eb05b28107f4365a8f3ff9b192ce72f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:22.718626 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e44f8-902f-4ad9-8b7e-3300d1073b08 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:22.720414 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 218/441] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:10:22 2022] GET /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:22.725751 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e44f8-902f-4ad9-8b7e-3300d1073b08 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:22.789445 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:22.817058 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a78fe-01d3-4771-a8ba-6f621da79642 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:22.831435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e44f8-902f-4ad9-8b7e-3300d1073b08 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:22.853469 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a74cd-4ddb-4ca0-aeec-86ef5c9482ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:22.854092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a74cd-4ddb-4ca0-aeec-86ef5c9482ce None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:22.854368 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a74cd-4ddb-4ca0-aeec-86ef5c9482ce None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:22.854636 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a74cd-4ddb-4ca0-aeec-86ef5c9482ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:22.864788 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7a74cd-4ddb-4ca0-aeec-86ef5c9482ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:22.866872 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29b18b-48ed-4d6a-b7ff-7404e58264e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:22.867494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29b18b-48ed-4d6a-b7ff-7404e58264e3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:22.867787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29b18b-48ed-4d6a-b7ff-7404e58264e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:22.868073 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29b18b-48ed-4d6a-b7ff-7404e58264e3 None admin] PATH_INFO: `/v3/groups/e68ae333760c4cffad23ec61ffad415a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:22.869823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29b18b-48ed-4d6a-b7ff-7404e58264e3 None admin] RBAC: Authorizing `identity:get_group(group_id=e68ae333760c4cffad23ec61ffad415a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:22.870136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-7b7a74cd-4ddb-4ca0-aeec-86ef5c9482ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:10:22.870136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:22.870136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:22.870136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:22.870136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:22.870136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:22.870136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:22.870136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:22.870136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:22.870136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:22.870136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:22.870136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:22.870136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:22.870136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:22.870136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:22.870136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:22.870966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:22.870966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:22.870966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:22.870966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:22.870966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:22.870966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:22.870966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:22.870966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:22.870966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:22.870966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:22.870966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:22.870966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:22.870966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:22.870966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:22.870966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:22.870966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:22.871834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:22.871834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:22.871834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:22.871834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:22.871834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:22.871834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:22.871834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:22.871834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:22.871834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:22.871834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:22.871834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:10:22.871834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:22.878123 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29b18b-48ed-4d6a-b7ff-7404e58264e3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:22.878123 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 219/442] 10.222.0.81 () {56 vars in 1180 bytes} [Mon Sep 19 11:10:22 2022] GET /identity/v3/groups/e68ae333760c4cffad23ec61ffad415a => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:22.878269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 224/443] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:10:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:10:22.878337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29b18b-48ed-4d6a-b7ff-7404e58264e3 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:22.878690 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a74cd-4ddb-4ca0-aeec-86ef5c9482ce None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:22.904468 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:22.975520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29b18b-48ed-4d6a-b7ff-7404e58264e3 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:22.976969 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a74cd-4ddb-4ca0-aeec-86ef5c9482ce None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:23.008968 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a92b3-c5c9-43ae-a5a9-6e5fbc718db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:23.009582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a92b3-c5c9-43ae-a5a9-6e5fbc718db3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:23.009863 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a92b3-c5c9-43ae-a5a9-6e5fbc718db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:23.010144 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a92b3-c5c9-43ae-a5a9-6e5fbc718db3 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:23.010619 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79933a39-d6b6-4066-9701-05491c42c70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:23.010802 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a92b3-c5c9-43ae-a5a9-6e5fbc718db3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:23.011106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a92b3-c5c9-43ae-a5a9-6e5fbc718db3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:23.011196 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79933a39-d6b6-4066-9701-05491c42c70e None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:23.011473 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79933a39-d6b6-4066-9701-05491c42c70e None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:23.011740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79933a39-d6b6-4066-9701-05491c42c70e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:23.012486 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79933a39-d6b6-4066-9701-05491c42c70e None admin] RBAC: Adding query filter params (group.id=e68ae333760c4cffad23ec61ffad415a, role.id=0c30c75e227c416da6e4814bce700e8d, scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:23.012588 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a92b3-c5c9-43ae-a5a9-6e5fbc718db3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:23.012792 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79933a39-d6b6-4066-9701-05491c42c70e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:23.014375 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79933a39-d6b6-4066-9701-05491c42c70e None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:23.017470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:23.020638 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 220/444] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:23.021646 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a92b3-c5c9-43ae-a5a9-6e5fbc718db3 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:23.026070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 225/445] 10.222.0.81 () {56 vars in 1376 bytes} [Mon Sep 19 11:10:22 2022] GET /identity/v3/role_assignments?group.id=e68ae333760c4cffad23ec61ffad415a&scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f&role.id=0c30c75e227c416da6e4814bce700e8d => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:23.027249 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79933a39-d6b6-4066-9701-05491c42c70e None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:23.134462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:23.144171 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79933a39-d6b6-4066-9701-05491c42c70e None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:23.152457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a92b3-c5c9-43ae-a5a9-6e5fbc718db3 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:23.182197 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6de76-efeb-4106-a15b-9c6299273a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:23.183081 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6de76-efeb-4106-a15b-9c6299273a2b None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:23.183362 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6de76-efeb-4106-a15b-9c6299273a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:23.184013 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6de76-efeb-4106-a15b-9c6299273a2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:23.196535 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38966289-f274-41c0-98e8-0b36b6737ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:23.197701 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38966289-f274-41c0-98e8-0b36b6737ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:23.197981 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38966289-f274-41c0-98e8-0b36b6737ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:23.198251 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38966289-f274-41c0-98e8-0b36b6737ca5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:23.211791 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38966289-f274-41c0-98e8-0b36b6737ca5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:23.219801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-38966289-f274-41c0-98e8-0b36b6737ca5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:10:23.219801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:23.219801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:23.219801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:23.219801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:23.219801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:23.219801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:23.219801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:23.219801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:23.219801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:23.219801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:23.219801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:23.219801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:23.219801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:23.219801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:23.219801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:23.220866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:23.220866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:23.220866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:23.220866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:23.220866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:23.220866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:23.220866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:23.220866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:23.220866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:23.220866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:23.220866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:23.220866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:23.220866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:23.220866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:23.220866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:23.220866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:23.222465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:23.222465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:23.222465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:23.222465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:23.222465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:23.222465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:23.222465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:23.222465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:23.222465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:23.222465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:23.222465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:10:23.222465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:23.222465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 221/446] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:10:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:23.226556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38966289-f274-41c0-98e8-0b36b6737ca5 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:23.259985 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:23.264823 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b6de76-efeb-4106-a15b-9c6299273a2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:23.266297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b6de76-efeb-4106-a15b-9c6299273a2b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:23.348661 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38966289-f274-41c0-98e8-0b36b6737ca5 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:23.385937 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 226/447] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:10:23.391421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ce7f0-2c61-445d-bbc6-397970daba67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:23.391684 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6de76-efeb-4106-a15b-9c6299273a2b None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:23.392072 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ce7f0-2c61-445d-bbc6-397970daba67 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:23.392354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ce7f0-2c61-445d-bbc6-397970daba67 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:23.392626 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ce7f0-2c61-445d-bbc6-397970daba67 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:23.393327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ce7f0-2c61-445d-bbc6-397970daba67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:23.393851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ce7f0-2c61-445d-bbc6-397970daba67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:23.396182 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ce7f0-2c61-445d-bbc6-397970daba67 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:23.406285 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 222/448] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:23.411880 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ce7f0-2c61-445d-bbc6-397970daba67 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:23.424546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:23.487780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:23.491266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6de76-efeb-4106-a15b-9c6299273a2b None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:23.510338 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ce7f0-2c61-445d-bbc6-397970daba67 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:23.524849 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625527e-77cb-41bc-b3ea-e7f4a36ffcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:23.525994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625527e-77cb-41bc-b3ea-e7f4a36ffcb9 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:23.526270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625527e-77cb-41bc-b3ea-e7f4a36ffcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:23.526533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625527e-77cb-41bc-b3ea-e7f4a36ffcb9 None admin] PATH_INFO: `/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:23.527645 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625527e-77cb-41bc-b3ea-e7f4a36ffcb9 None admin] RBAC: Authorizing `identity:get_project(project_id=8eb05b28107f4365a8f3ff9b192ce72f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:23.529098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625527e-77cb-41bc-b3ea-e7f4a36ffcb9 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:23.530869 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 227/449] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:10:23 2022] GET /identity/v3/projects/8eb05b28107f4365a8f3ff9b192ce72f => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:23.536347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625527e-77cb-41bc-b3ea-e7f4a36ffcb9 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:23.543448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0c7f2-ce93-4844-9968-663590e6d1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:23.544076 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0c7f2-ce93-4844-9968-663590e6d1fb None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:23.544354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0c7f2-ce93-4844-9968-663590e6d1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:23.544628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0c7f2-ce93-4844-9968-663590e6d1fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:23.612340 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0c7f2-ce93-4844-9968-663590e6d1fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:23.613811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0c7f2-ce93-4844-9968-663590e6d1fb None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:23.624875 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:23.632976 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625527e-77cb-41bc-b3ea-e7f4a36ffcb9 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:23.673116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769f5ea-e4dc-47aa-9c71-d55949fe4d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:23.673743 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769f5ea-e4dc-47aa-9c71-d55949fe4d73 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:23.674023 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769f5ea-e4dc-47aa-9c71-d55949fe4d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:23.674294 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769f5ea-e4dc-47aa-9c71-d55949fe4d73 None admin] PATH_INFO: `/v3/groups/e68ae333760c4cffad23ec61ffad415a` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:23.675403 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769f5ea-e4dc-47aa-9c71-d55949fe4d73 None admin] RBAC: Authorizing `identity:get_group(group_id=e68ae333760c4cffad23ec61ffad415a)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:23.676872 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769f5ea-e4dc-47aa-9c71-d55949fe4d73 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:23.678441 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 228/450] 10.222.0.81 () {56 vars in 1180 bytes} [Mon Sep 19 11:10:23 2022] GET /identity/v3/groups/e68ae333760c4cffad23ec61ffad415a => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:23.684166 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769f5ea-e4dc-47aa-9c71-d55949fe4d73 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:23.712074 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 223/451] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 11:10:23.718154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0c7f2-ce93-4844-9968-663590e6d1fb None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:23.719231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:23.783197 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769f5ea-e4dc-47aa-9c71-d55949fe4d73 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:23.813210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0c7f2-ce93-4844-9968-663590e6d1fb None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:23.816300 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e48569-18fe-47f2-8c8c-7e0178500dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:23.816892 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e48569-18fe-47f2-8c8c-7e0178500dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:23.817168 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e48569-18fe-47f2-8c8c-7e0178500dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:23.817436 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e48569-18fe-47f2-8c8c-7e0178500dd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:23.818149 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e48569-18fe-47f2-8c8c-7e0178500dd3 None admin] RBAC: Adding query filter params (group.id=e68ae333760c4cffad23ec61ffad415a, role.id=4cc230d31ac94f4c89bf827491c01bd7, scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:23.818452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e48569-18fe-47f2-8c8c-7e0178500dd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:23.819890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e48569-18fe-47f2-8c8c-7e0178500dd3 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:23.824016 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:23.826171 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 229/452] 10.222.0.81 () {56 vars in 1376 bytes} [Mon Sep 19 11:10:23 2022] GET /identity/v3/role_assignments?group.id=e68ae333760c4cffad23ec61ffad415a&scope.project.id=8eb05b28107f4365a8f3ff9b192ce72f&role.id=4cc230d31ac94f4c89bf827491c01bd7 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:23.849677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34997c-3bda-4098-bfd9-109d6d4b1d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:23.850328 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34997c-3bda-4098-bfd9-109d6d4b1d43 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:23.850610 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34997c-3bda-4098-bfd9-109d6d4b1d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:23.850884 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34997c-3bda-4098-bfd9-109d6d4b1d43 None admin] PATH_INFO: `/v3/projects/5d5c1387f77e4df7ac9b4d83d404bc68` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:23.852094 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34997c-3bda-4098-bfd9-109d6d4b1d43 None admin] RBAC: Authorizing `identity:get_project(project_id=5d5c1387f77e4df7ac9b4d83d404bc68)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:23.853618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34997c-3bda-4098-bfd9-109d6d4b1d43 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:23.855507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 224/453] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:10:23 2022] GET /identity/v3/projects/5d5c1387f77e4df7ac9b4d83d404bc68 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:23.861130 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e48569-18fe-47f2-8c8c-7e0178500dd3 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:23.975484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:23.978830 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e48569-18fe-47f2-8c8c-7e0178500dd3 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:24.013355 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237101a-b793-469f-950a-942c5f66b6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:24.013992 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237101a-b793-469f-950a-942c5f66b6fd None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:24.014271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237101a-b793-469f-950a-942c5f66b6fd None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:24.014548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237101a-b793-469f-950a-942c5f66b6fd None admin] PATH_INFO: `/v3/groups/337ef65b0777469d82e8a49b4a1d7a3b` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:24.015668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237101a-b793-469f-950a-942c5f66b6fd None admin] RBAC: Authorizing `identity:get_group(group_id=337ef65b0777469d82e8a49b4a1d7a3b)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:24.017162 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237101a-b793-469f-950a-942c5f66b6fd None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:24.018730 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 230/454] 10.222.0.81 () {56 vars in 1180 bytes} [Mon Sep 19 11:10:23 2022] GET /identity/v3/groups/337ef65b0777469d82e8a49b4a1d7a3b => generated 229 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:24.024749 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34997c-3bda-4098-bfd9-109d6d4b1d43 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:24.126397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34997c-3bda-4098-bfd9-109d6d4b1d43 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:24.161818 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e5377-a383-4d29-a650-f553657272c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:24.162419 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e5377-a383-4d29-a650-f553657272c9 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:24.162699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e5377-a383-4d29-a650-f553657272c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:24.162976 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e5377-a383-4d29-a650-f553657272c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:24.163697 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e5377-a383-4d29-a650-f553657272c9 None admin] RBAC: Adding query filter params (group.id=337ef65b0777469d82e8a49b4a1d7a3b, role.id=753f8b35d5f0458d8f62d0ad6e28033f, scope.project.id=5d5c1387f77e4df7ac9b4d83d404bc68) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:24.164089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e5377-a383-4d29-a650-f553657272c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:24.165531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e5377-a383-4d29-a650-f553657272c9 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:24.172133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 225/455] 10.222.0.81 () {56 vars in 1376 bytes} [Mon Sep 19 11:10:24 2022] GET /identity/v3/role_assignments?group.id=337ef65b0777469d82e8a49b4a1d7a3b&scope.project.id=5d5c1387f77e4df7ac9b4d83d404bc68&role.id=753f8b35d5f0458d8f62d0ad6e28033f => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:26.701756 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c53148-f7f9-4cd4-b4cc-9cab5ca6a660 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:26.702270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c53148-f7f9-4cd4-b4cc-9cab5ca6a660 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:26.702270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c53148-f7f9-4cd4-b4cc-9cab5ca6a660 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:26.704059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 231/456] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:26.709355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26b974-8a4a-433e-affd-795e339c3f8f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:26.709789 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26b974-8a4a-433e-affd-795e339c3f8f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:26.709968 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26b974-8a4a-433e-affd-795e339c3f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:26.732733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6f26b974-8a4a-433e-affd-795e339c3f8f None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:26.733950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6f26b974-8a4a-433e-affd-795e339c3f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:26.782132 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:26.826590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 226/457] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:27.121770 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad7de6-ac0c-488f-8c70-e80df9ffe581 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:27.122472 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad7de6-ac0c-488f-8c70-e80df9ffe581 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:27.122961 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad7de6-ac0c-488f-8c70-e80df9ffe581 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:27.124822 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 232/458] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:27.129644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ee304-fcfe-4159-b598-0037cea0c6cf None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:27.130146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ee304-fcfe-4159-b598-0037cea0c6cf None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:27.130621 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ee304-fcfe-4159-b598-0037cea0c6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:27.156708 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e84ee304-fcfe-4159-b598-0037cea0c6cf None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:27.158256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e84ee304-fcfe-4159-b598-0037cea0c6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:27.197663 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de91e7-a0fd-4866-90ca-3f45adf0f225 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:27.198310 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de91e7-a0fd-4866-90ca-3f45adf0f225 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:27.198784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de91e7-a0fd-4866-90ca-3f45adf0f225 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:27.200595 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 233/459] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:27.206135 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d9398-c330-4b00-b468-c2292cb613b8 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:27.206646 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d9398-c330-4b00-b468-c2292cb613b8 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:27.207138 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d9398-c330-4b00-b468-c2292cb613b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:27.237763 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 227/460] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:27.257003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-231d9398-c330-4b00-b468-c2292cb613b8 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:27.258581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-231d9398-c330-4b00-b468-c2292cb613b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:27.318947 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ee304-fcfe-4159-b598-0037cea0c6cf None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:27.381509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 234/461] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:10:27.385929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c80e3c-6a06-4541-a689-daa01bcfda8e None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:27.386212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c80e3c-6a06-4541-a689-daa01bcfda8e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:27.386485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c80e3c-6a06-4541-a689-daa01bcfda8e None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:27.388179 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 235/462] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:27.394384 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f9cee-c3fd-4a40-8b0b-fe0562ce1287 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:27.395740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f9cee-c3fd-4a40-8b0b-fe0562ce1287 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:27.396058 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f9cee-c3fd-4a40-8b0b-fe0562ce1287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:27.430804 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-589f9cee-c3fd-4a40-8b0b-fe0562ce1287 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:27.432551 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-589f9cee-c3fd-4a40-8b0b-fe0562ce1287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:27.469465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ee304-fcfe-4159-b598-0037cea0c6cf None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:27.471457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:27.520017 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b568d7-d742-400a-93c6-55c47b3ac83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:27.521025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b568d7-d742-400a-93c6-55c47b3ac83e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:27.521506 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b568d7-d742-400a-93c6-55c47b3ac83e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:27.521976 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b568d7-d742-400a-93c6-55c47b3ac83e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:27.534835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b568d7-d742-400a-93c6-55c47b3ac83e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:27.536975 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b568d7-d742-400a-93c6-55c47b3ac83e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:27.556653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-a4b568d7-d742-400a-93c6-55c47b3ac83e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:27.556653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:27.556653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:27.556653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:27.556653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:27.556653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:27.556653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:27.556653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:27.556653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:27.556653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:27.556653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:27.556653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:27.556653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:27.556653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:27.556653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:27.556653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:27.557590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:27.557590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:27.557590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:27.557590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:27.557590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:27.557590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:27.557590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:27.557590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:27.557590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:27.557590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:27.557590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:27.557590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:27.557590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:27.557590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:27.557590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:27.557590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:27.557590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:27.558466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:27.558466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:27.558466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:27.559187 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 236/463] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:27.560044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f9cee-c3fd-4a40-8b0b-fe0562ce1287 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:27.568154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 228/464] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:10:27.578312 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b568d7-d742-400a-93c6-55c47b3ac83e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:27.619842 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:27.724290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f9cee-c3fd-4a40-8b0b-fe0562ce1287 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:27.739000 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b568d7-d742-400a-93c6-55c47b3ac83e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:27.768800 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb5e9a-c62e-4a22-9478-06f597367795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:27.769706 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb5e9a-c62e-4a22-9478-06f597367795 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:27.769989 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb5e9a-c62e-4a22-9478-06f597367795 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:27.770264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb5e9a-c62e-4a22-9478-06f597367795 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:27.777467 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39cf75a-4d36-4db6-8216-8b5df970d6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:27.778298 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cf75a-4d36-4db6-8216-8b5df970d6ac None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:27.778784 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cf75a-4d36-4db6-8216-8b5df970d6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:27.779159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cf75a-4d36-4db6-8216-8b5df970d6ac None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:27.780012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39cf75a-4d36-4db6-8216-8b5df970d6ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:27.780505 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39cf75a-4d36-4db6-8216-8b5df970d6ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:27.782061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39cf75a-4d36-4db6-8216-8b5df970d6ac None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:27.782723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb5e9a-c62e-4a22-9478-06f597367795 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:27.784618 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb5e9a-c62e-4a22-9478-06f597367795 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:27.799448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 229/465] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:27.800736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39cf75a-4d36-4db6-8216-8b5df970d6ac None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:27.802115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-9afb5e9a-c62e-4a22-9478-06f597367795 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:27.802115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:27.802115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:27.802115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:27.802115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:27.802115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:27.802115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:27.802115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:27.802115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:27.802115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:27.802115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:27.802115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:27.802115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:27.802115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:27.802115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:27.802115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:27.802973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:27.802973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:27.802973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:27.802973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:27.802973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:27.802973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:27.802973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:27.802973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:27.802973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:27.802973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:27.802973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:27.802973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:27.802973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:27.802973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:27.802973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:27.802973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:27.802973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:27.803845 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:27.803845 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:27.803845 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:27.803845 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 237/466] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:27.804070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb5e9a-c62e-4a22-9478-06f597367795 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:27.820679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:27.911075 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39cf75a-4d36-4db6-8216-8b5df970d6ac None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:27.914849 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:27.937886 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb5e9a-c62e-4a22-9478-06f597367795 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:27.951498 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a467655-14b5-46a4-99e9-50efe921b794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:27.953978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a467655-14b5-46a4-99e9-50efe921b794 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:27.954282 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a467655-14b5-46a4-99e9-50efe921b794 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:27.955465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a467655-14b5-46a4-99e9-50efe921b794 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:27.966352 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a467655-14b5-46a4-99e9-50efe921b794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:27.967848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a467655-14b5-46a4-99e9-50efe921b794 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:27.979464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab354ed-99ae-43cf-b271-e0a823b76eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:27.980103 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab354ed-99ae-43cf-b271-e0a823b76eea None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:27.980386 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab354ed-99ae-43cf-b271-e0a823b76eea None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:27.980659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab354ed-99ae-43cf-b271-e0a823b76eea None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:27.981302 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab354ed-99ae-43cf-b271-e0a823b76eea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:27.982775 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab354ed-99ae-43cf-b271-e0a823b76eea None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:27.983475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-3a467655-14b5-46a4-99e9-50efe921b794 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:27.983475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:27.983475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:27.983475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:27.983475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:27.983475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:27.983475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:27.983475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:27.983475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:27.983475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:27.983475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:27.983475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:27.983475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:27.983475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:27.983475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:27.983475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:27.984342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:27.984342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:27.984342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:27.984342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:27.984342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:27.984342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:27.984342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:27.984342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:27.984342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:27.984342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:27.984342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:27.984342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:27.984342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:27.984342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:27.984342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:27.984342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:27.984342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:27.985204 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:27.985204 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:27.985204 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:27.985204 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 230/467] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:27.986319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a467655-14b5-46a4-99e9-50efe921b794 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:28.007339 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 238/468] 10.222.0.81 () {60 vars in 1132 bytes} [Mon Sep 19 11:10:27 2022] POST /identity/v3/users => generated 246 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:10:28.008537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab354ed-99ae-43cf-b271-e0a823b76eea None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:28.075431 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:28.128854 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a467655-14b5-46a4-99e9-50efe921b794 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:28.142342 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab354ed-99ae-43cf-b271-e0a823b76eea None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:28.180590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7a8a9-16e9-40a1-a6ea-29c9e5543326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:28.181275 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7a8a9-16e9-40a1-a6ea-29c9e5543326 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:28.181559 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7a8a9-16e9-40a1-a6ea-29c9e5543326 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:28.181833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7a8a9-16e9-40a1-a6ea-29c9e5543326 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:28.182494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd7a8a9-16e9-40a1-a6ea-29c9e5543326 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:28.182801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd7a8a9-16e9-40a1-a6ea-29c9e5543326 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:28.184279 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd7a8a9-16e9-40a1-a6ea-29c9e5543326 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:28.193071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b4cce-a86d-4391-965a-5a2c62b4b541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:28.194655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b4cce-a86d-4391-965a-5a2c62b4b541 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:28.195553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b4cce-a86d-4391-965a-5a2c62b4b541 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:28.196075 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b4cce-a86d-4391-965a-5a2c62b4b541 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:28.199352 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 231/469] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:28.200282 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:28.201353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7a8a9-16e9-40a1-a6ea-29c9e5543326 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:28.213537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b4cce-a86d-4391-965a-5a2c62b4b541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:28.215096 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b4cce-a86d-4391-965a-5a2c62b4b541 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:28.236559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-762b4cce-a86d-4391-965a-5a2c62b4b541 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:28.236559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:28.236559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:28.236559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:28.236559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:28.236559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:28.236559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:28.236559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:28.236559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:28.236559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:28.236559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:28.236559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:28.236559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:28.236559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:28.236559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:28.236559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:28.237501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:28.237501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:28.237501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:28.237501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:28.237501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:28.237501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:28.237501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:28.237501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:28.237501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:28.237501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:28.237501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:28.237501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:28.237501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:28.237501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:28.237501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:28.237501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:28.237501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:28.238634 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:28.238634 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:28.238634 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:28.238634 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 239/470] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:28.238634 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b4cce-a86d-4391-965a-5a2c62b4b541 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:28.329869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7a8a9-16e9-40a1-a6ea-29c9e5543326 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:28.363781 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b4cce-a86d-4391-965a-5a2c62b4b541 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:28.364553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:28.375510 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e50e0-e052-40ee-aee6-0ecee689a649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:28.376377 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e50e0-e052-40ee-aee6-0ecee689a649 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:28.376804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e50e0-e052-40ee-aee6-0ecee689a649 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:28.377251 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e50e0-e052-40ee-aee6-0ecee689a649 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:28.378053 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e50e0-e052-40ee-aee6-0ecee689a649 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:28.378569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e50e0-e052-40ee-aee6-0ecee689a649 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:28.380232 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e50e0-e052-40ee-aee6-0ecee689a649 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:28.392814 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 232/471] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:28.394116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e50e0-e052-40ee-aee6-0ecee689a649 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:28.408043 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d204b04b-ba3f-4582-b43d-7544222dc02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:28.408554 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204b04b-ba3f-4582-b43d-7544222dc02b None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:28.408838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204b04b-ba3f-4582-b43d-7544222dc02b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:28.409116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204b04b-ba3f-4582-b43d-7544222dc02b None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:28.409783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d204b04b-ba3f-4582-b43d-7544222dc02b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:28.411278 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d204b04b-ba3f-4582-b43d-7544222dc02b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:28.440711 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 240/472] 10.222.0.81 () {60 vars in 1132 bytes} [Mon Sep 19 11:10:28 2022] POST /identity/v3/users => generated 248 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:10:28.441649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d204b04b-ba3f-4582-b43d-7544222dc02b None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:28.505348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:28.518054 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e50e0-e052-40ee-aee6-0ecee689a649 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:28.573472 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bae41c-8ffe-412e-949d-c15ded53e37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:28.574111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bae41c-8ffe-412e-949d-c15ded53e37e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:28.574388 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bae41c-8ffe-412e-949d-c15ded53e37e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:28.574662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bae41c-8ffe-412e-949d-c15ded53e37e None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:28.575657 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bae41c-8ffe-412e-949d-c15ded53e37e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:28.575992 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bae41c-8ffe-412e-949d-c15ded53e37e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:28.578281 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bae41c-8ffe-412e-949d-c15ded53e37e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:28.583912 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d204b04b-ba3f-4582-b43d-7544222dc02b None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:28.596284 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 233/473] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:28.603835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bae41c-8ffe-412e-949d-c15ded53e37e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:28.637328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3dec96-60ee-4267-ad48-bbd75cae934f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:28.638448 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3dec96-60ee-4267-ad48-bbd75cae934f None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:28.638733 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3dec96-60ee-4267-ad48-bbd75cae934f None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:28.639005 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3dec96-60ee-4267-ad48-bbd75cae934f None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:28.640027 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3dec96-60ee-4267-ad48-bbd75cae934f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:28.642379 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3dec96-60ee-4267-ad48-bbd75cae934f None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:28.681043 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:28.686467 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 241/474] 10.222.0.81 () {60 vars in 1132 bytes} [Mon Sep 19 11:10:28 2022] POST /identity/v3/users => generated 248 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:10:28.686467 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e21137-e195-4dc0-82df-10cfb58026c5 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:28.686467 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e21137-e195-4dc0-82df-10cfb58026c5 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:28.686467 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e21137-e195-4dc0-82df-10cfb58026c5 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:28.691548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 242/475] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:28.695346 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c0950-19bf-48b3-94d4-2fe1a0d160b4 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:28.695620 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c0950-19bf-48b3-94d4-2fe1a0d160b4 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:28.696040 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c0950-19bf-48b3-94d4-2fe1a0d160b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:28.724578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-282c0950-19bf-48b3-94d4-2fe1a0d160b4 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:28.726128 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-282c0950-19bf-48b3-94d4-2fe1a0d160b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:28.769926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bae41c-8ffe-412e-949d-c15ded53e37e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:28.817723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393088f-8aa2-44da-8ef7-1828398017c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:28.818435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393088f-8aa2-44da-8ef7-1828398017c0 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:28.818633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393088f-8aa2-44da-8ef7-1828398017c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:28.818893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393088f-8aa2-44da-8ef7-1828398017c0 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:28.819564 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393088f-8aa2-44da-8ef7-1828398017c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:28.821080 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393088f-8aa2-44da-8ef7-1828398017c0 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:28.829442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 243/476] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:28.857068 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 234/477] 10.222.0.81 () {60 vars in 1132 bytes} [Mon Sep 19 11:10:28 2022] POST /identity/v3/users => generated 249 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:10:29.129912 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c0950-19bf-48b3-94d4-2fe1a0d160b4 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:29.163883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:29.242366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c0950-19bf-48b3-94d4-2fe1a0d160b4 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:29.281771 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad18f33-1321-4b8e-81d5-dd66ce9e289c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:29.282386 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad18f33-1321-4b8e-81d5-dd66ce9e289c None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:29.282669 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad18f33-1321-4b8e-81d5-dd66ce9e289c None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:29.282939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad18f33-1321-4b8e-81d5-dd66ce9e289c None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:29.283622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad18f33-1321-4b8e-81d5-dd66ce9e289c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:29.285038 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad18f33-1321-4b8e-81d5-dd66ce9e289c None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:29.295996 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.sql.core [None req-cad18f33-1321-4b8e-81d5-dd66ce9e289c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 19 11:10:29.295996 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 19 11:10:29.295996 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [parameters: {'id': '1f93a86b2fd84e2381b67f47a6a11d38', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 19 11:10:29.295996 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81461) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 19 11:10:29.313292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-cad18f33-1321-4b8e-81d5-dd66ce9e289c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 19 11:10:29.313292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:29.313292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 11:10:29.313292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 11:10:29.313292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 11:10:29.313292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 11:10:29.313292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 11:10:29.313292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 11:10:29.313292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 11:10:29.313292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application conn.query(q) Sep 19 11:10:29.313292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 11:10:29.313292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 11:10:29.313292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 11:10:29.313292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application result.read() Sep 19 11:10:29.313292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 11:10:29.314194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 11:10:29.314194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 11:10:29.314194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 11:10:29.314194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 11:10:29.314194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 11:10:29.314194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 11:10:29.314194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 11:10:29.314194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 19 11:10:29.314194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:29.314194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 19 11:10:29.314194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:29.314194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:29.314194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 19 11:10:29.314194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 19 11:10:29.314194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 19 11:10:29.314194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return ref.to_dict() Sep 19 11:10:29.314194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 11:10:29.314194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application next(self.gen) Sep 19 11:10:29.315107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 19 11:10:29.315107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application yield resource Sep 19 11:10:29.315107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 11:10:29.315107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application next(self.gen) Sep 19 11:10:29.315107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 19 11:10:29.315107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self.session.rollback() Sep 19 11:10:29.315107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 11:10:29.315107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self.force_reraise() Sep 19 11:10:29.315107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 11:10:29.315107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise self.value Sep 19 11:10:29.315107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 19 11:10:29.315107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 19 11:10:29.315107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 19 11:10:29.315107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application session.commit() Sep 19 11:10:29.315107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 19 11:10:29.315107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 19 11:10:29.315107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 19 11:10:29.316044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._prepare_impl() Sep 19 11:10:29.316044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 19 11:10:29.316044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self.session.flush() Sep 19 11:10:29.316044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 19 11:10:29.316044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._flush(objects) Sep 19 11:10:29.316044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 19 11:10:29.316044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 19 11:10:29.316044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 19 11:10:29.316044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application compat.raise_( Sep 19 11:10:29.316044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 11:10:29.316044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception Sep 19 11:10:29.316044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 19 11:10:29.316044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application flush_context.execute() Sep 19 11:10:29.316044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 19 11:10:29.316044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rec.execute(self) Sep 19 11:10:29.316044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 19 11:10:29.316044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 19 11:10:29.316926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 19 11:10:29.316926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 19 11:10:29.316926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 19 11:10:29.316926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 19 11:10:29.316926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 19 11:10:29.316926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 19 11:10:29.316926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 19 11:10:29.316926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 19 11:10:29.316926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 19 11:10:29.316926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 19 11:10:29.316926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 19 11:10:29.316926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 19 11:10:29.316926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 19 11:10:29.316926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 19 11:10:29.316926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 11:10:29.316926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception Sep 19 11:10:29.317742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 11:10:29.317742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 11:10:29.317742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 11:10:29.317742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 11:10:29.317742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 11:10:29.317742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 11:10:29.317742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 11:10:29.317742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application conn.query(q) Sep 19 11:10:29.317742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 11:10:29.317742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 11:10:29.317742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 11:10:29.317742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application result.read() Sep 19 11:10:29.317742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 11:10:29.317742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 11:10:29.317742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 11:10:29.317742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 11:10:29.318611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 11:10:29.318611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 11:10:29.318611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 11:10:29.318611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 11:10:29.318611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 19 11:10:29.318611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 19 11:10:29.318611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [parameters: {'id': '1f93a86b2fd84e2381b67f47a6a11d38', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 19 11:10:29.318611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 19 11:10:29.318611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:29.318611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:10:29.318611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:29.318611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:29.318611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:29.318611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:29.318611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:29.318611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:29.319419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:29.319419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:29.319419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:29.319419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:29.319419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:29.319419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:29.319419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 19 11:10:29.319419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 19 11:10:29.319419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:29.319419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:29.319419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 19 11:10:29.319419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 19 11:10:29.319419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 19 11:10:29.319419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 19 11:10:29.319419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 19 11:10:29.319419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:29.323388 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 244/478] 10.222.0.81 () {60 vars in 1131 bytes} [Mon Sep 19 11:10:29 2022] POST /identity/v3/roles => generated 144 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:29.323523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393088f-8aa2-44da-8ef7-1828398017c0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:29.467076 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393088f-8aa2-44da-8ef7-1828398017c0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:29.469034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:29.505676 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8d61e-bcf8-4c0e-9274-2599cdcad214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:29.506301 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8d61e-bcf8-4c0e-9274-2599cdcad214 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:29.506577 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8d61e-bcf8-4c0e-9274-2599cdcad214 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:29.506851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8d61e-bcf8-4c0e-9274-2599cdcad214 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:29.518036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8d61e-bcf8-4c0e-9274-2599cdcad214 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:29.522807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-75b8d61e-bcf8-4c0e-9274-2599cdcad214 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:10:29.522807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:29.522807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:29.522807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:29.522807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:29.522807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:29.522807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:29.522807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:29.522807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:29.522807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:29.522807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:29.522807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:29.522807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:29.522807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:29.522807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:29.522807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:29.523702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:29.523702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:29.523702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:29.523702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:29.523702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:29.523702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:29.523702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:29.523702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:29.523702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:29.523702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:29.523702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:29.523702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:29.523702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:29.523702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:29.523702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:29.523702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:29.524548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:29.524548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:29.524548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:29.524548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:29.524548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:29.524548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:29.524548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:29.524548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:29.524548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:29.524548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:29.524548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 11:10:29.524548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:29.524548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 235/479] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:10:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:29.532530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad18f33-1321-4b8e-81d5-dd66ce9e289c None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:29.664995 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:29.674667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad18f33-1321-4b8e-81d5-dd66ce9e289c None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:29.727489 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b28ba9-eb0c-48ba-8d18-b8161b9d92e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:29.729727 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b28ba9-eb0c-48ba-8d18-b8161b9d92e4 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:29.730018 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b28ba9-eb0c-48ba-8d18-b8161b9d92e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:29.730300 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b28ba9-eb0c-48ba-8d18-b8161b9d92e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:29.730989 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b28ba9-eb0c-48ba-8d18-b8161b9d92e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:29.731737 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b28ba9-eb0c-48ba-8d18-b8161b9d92e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:29.733706 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b28ba9-eb0c-48ba-8d18-b8161b9d92e4 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:29.747865 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 245/480] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:30.469123 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14cc73-35e0-452d-aad1-95f2c09953f4 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:30.470320 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14cc73-35e0-452d-aad1-95f2c09953f4 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:30.470823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14cc73-35e0-452d-aad1-95f2c09953f4 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:30.472628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 236/481] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:30.477547 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd5f52-e383-4687-bca2-58ff7903e3ce None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:30.478570 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd5f52-e383-4687-bca2-58ff7903e3ce None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:30.479054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd5f52-e383-4687-bca2-58ff7903e3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:30.504740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-1afd5f52-e383-4687-bca2-58ff7903e3ce None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:30.506302 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-1afd5f52-e383-4687-bca2-58ff7903e3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:30.592969 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 246/482] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:10:30.985600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14cc73-35e0-452d-aad1-95f2c09953f4 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:31.133617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14cc73-35e0-452d-aad1-95f2c09953f4 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:31.184195 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d61ff-7713-4e6d-bc7e-14cecbc100da None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:31.184736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f56173-4b08-4525-915f-ac01c556d6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:31.184831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d61ff-7713-4e6d-bc7e-14cecbc100da None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:31.184831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d61ff-7713-4e6d-bc7e-14cecbc100da None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:31.184987 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f56173-4b08-4525-915f-ac01c556d6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:31.185255 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f56173-4b08-4525-915f-ac01c556d6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:31.185351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f56173-4b08-4525-915f-ac01c556d6e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:31.186315 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 247/483] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:10:31.191330 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d876ef-d2f0-40ef-88d4-4b91a64dc5ae None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:31.191713 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d876ef-d2f0-40ef-88d4-4b91a64dc5ae None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:31.192021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d876ef-d2f0-40ef-88d4-4b91a64dc5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:31.207335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f56173-4b08-4525-915f-ac01c556d6e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:31.217082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-75f56173-4b08-4525-915f-ac01c556d6e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:31.217082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:31.217082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:31.217082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:31.217082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:31.217082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:31.217082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:31.217082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:31.217082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:31.217082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:31.217082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:31.217082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:31.217082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:31.217082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:31.217082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:31.217082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:31.218623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:31.218623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:31.218623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:31.218623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:31.218623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:31.218623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:31.218623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:31.218623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:31.218623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:31.218623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:31.218623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:31.218623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:31.218623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:31.218623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:31.218623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:31.218623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:31.220036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:31.220036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:31.220036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:31.220036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:31.220036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:31.220036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:31.220036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:31.220036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:31.220036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:31.220036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:31.220036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:31.220036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:31.220036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 237/484] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:10:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:31.224853 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-80d876ef-d2f0-40ef-88d4-4b91a64dc5ae None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:31.226070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-80d876ef-d2f0-40ef-88d4-4b91a64dc5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:31.228776 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f56173-4b08-4525-915f-ac01c556d6e9 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:31.273889 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:31.298983 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 248/485] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:10:31.308366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06783a9-0488-4893-98ad-ef73791a61ca None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:31.308665 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06783a9-0488-4893-98ad-ef73791a61ca None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:31.309006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06783a9-0488-4893-98ad-ef73791a61ca None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:31.314397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 249/486] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:31 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:31.319990 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d73f1-6991-45ca-9606-c040e15bad18 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:31.320278 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d73f1-6991-45ca-9606-c040e15bad18 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:31.320577 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d73f1-6991-45ca-9606-c040e15bad18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:31.357359 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-e82d73f1-6991-45ca-9606-c040e15bad18 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:31.358567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e82d73f1-6991-45ca-9606-c040e15bad18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:31.361608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f56173-4b08-4525-915f-ac01c556d6e9 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:31.378051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:31.401128 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ef6ce-9f12-4d1b-ab35-67a274195f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:31.402119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ef6ce-9f12-4d1b-ab35-67a274195f72 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:31.402642 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ef6ce-9f12-4d1b-ab35-67a274195f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:31.403095 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ef6ce-9f12-4d1b-ab35-67a274195f72 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:31.404112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ef6ce-9f12-4d1b-ab35-67a274195f72 None admin] RBAC: Adding query filter params (name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:31.404627 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ef6ce-9f12-4d1b-ab35-67a274195f72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:31.406384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ef6ce-9f12-4d1b-ab35-67a274195f72 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:31.415992 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 238/487] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:10:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:31.422003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ef6ce-9f12-4d1b-ab35-67a274195f72 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:31.494042 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 250/488] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:10:31.543487 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:31.561186 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ef6ce-9f12-4d1b-ab35-67a274195f72 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:31.600981 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408fdc6-cf2c-4d0f-808b-68a6c7b51872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:31.601611 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408fdc6-cf2c-4d0f-808b-68a6c7b51872 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:31.601893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408fdc6-cf2c-4d0f-808b-68a6c7b51872 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:31.602168 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408fdc6-cf2c-4d0f-808b-68a6c7b51872 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:31.617869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408fdc6-cf2c-4d0f-808b-68a6c7b51872 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:31.619891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408fdc6-cf2c-4d0f-808b-68a6c7b51872 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:31.638451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-3408fdc6-cf2c-4d0f-808b-68a6c7b51872 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:31.638451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:31.638451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:31.638451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:31.638451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:31.638451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:31.638451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:31.638451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:31.638451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:31.638451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:31.638451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:31.638451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:31.638451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:31.638451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:31.638451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:31.638451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:31.639355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:31.639355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:31.639355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:31.639355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:31.639355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:31.639355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:31.639355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:31.639355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:31.639355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:31.639355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:31.639355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:31.639355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:31.639355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:31.639355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:31.639355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:31.639355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:31.639355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:31.640905 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:31.640905 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:31.640905 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:31.640905 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 239/489] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:31.645205 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d73f1-6991-45ca-9606-c040e15bad18 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:31.713132 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408fdc6-cf2c-4d0f-808b-68a6c7b51872 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:31.789434 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d73f1-6991-45ca-9606-c040e15bad18 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:31.831554 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74178611-ea36-4230-99c4-3fa62444780e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:31.833374 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74178611-ea36-4230-99c4-3fa62444780e None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:31.834538 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74178611-ea36-4230-99c4-3fa62444780e None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:31.836370 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74178611-ea36-4230-99c4-3fa62444780e None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:31.838475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74178611-ea36-4230-99c4-3fa62444780e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:31.839615 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74178611-ea36-4230-99c4-3fa62444780e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:31.841396 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74178611-ea36-4230-99c4-3fa62444780e None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:31.865886 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 251/490] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:31.871736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74178611-ea36-4230-99c4-3fa62444780e None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:31.881933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:31.903784 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408fdc6-cf2c-4d0f-808b-68a6c7b51872 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:31.938704 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38963ab-6a7d-4c0e-83e3-d8c8b5bae1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:31.939329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38963ab-6a7d-4c0e-83e3-d8c8b5bae1e6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:31.939606 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38963ab-6a7d-4c0e-83e3-d8c8b5bae1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:31.939901 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38963ab-6a7d-4c0e-83e3-d8c8b5bae1e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:31.942671 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:31.952486 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38963ab-6a7d-4c0e-83e3-d8c8b5bae1e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:31.957612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-a38963ab-6a7d-4c0e-83e3-d8c8b5bae1e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:31.957612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:31.957612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:31.957612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:31.957612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:31.957612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:31.957612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:31.957612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:31.957612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:31.957612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:31.957612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:31.957612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:31.957612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:31.957612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:31.957612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:31.957612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:31.958603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:31.958603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:31.958603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:31.958603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:31.958603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:31.958603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:31.958603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:31.958603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:31.958603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:31.958603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:31.958603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:31.958603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:31.958603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:31.958603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:31.958603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:31.958603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:31.959883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:31.959883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:31.959883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:31.959883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:31.959883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:31.959883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:31.959883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:31.959883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:31.959883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:31.959883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:31.959883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:31.959883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:31.959883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 240/491] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:10:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:10:31.965007 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38963ab-6a7d-4c0e-83e3-d8c8b5bae1e6 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:31.995556 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74178611-ea36-4230-99c4-3fa62444780e None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:32.034731 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3bdb2-a5a3-4164-95ef-5dcccdb8e429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:32.035978 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3bdb2-a5a3-4164-95ef-5dcccdb8e429 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:32.036470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3bdb2-a5a3-4164-95ef-5dcccdb8e429 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:32.036939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3bdb2-a5a3-4164-95ef-5dcccdb8e429 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:32.045971 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3bdb2-a5a3-4164-95ef-5dcccdb8e429 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:32.047655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3bdb2-a5a3-4164-95ef-5dcccdb8e429 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:32.062137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-2ff3bdb2-a5a3-4164-95ef-5dcccdb8e429 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:10:32.062137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:32.062137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:32.062137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:32.062137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:32.062137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:32.062137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:32.062137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:32.062137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:32.062137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:32.062137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:32.062137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:32.062137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:10:32.062137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:10:32.062137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:10:32.062137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:10:32.063079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:32.063079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:32.063079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:10:32.063079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:32.063079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:10:32.063079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:32.063079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:32.063079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:32.063079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:32.063079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:32.063079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:32.063079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:32.063079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:32.063079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:32.063079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:32.063079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:32.063079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:32.063079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:32.064015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:32.064015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:32.064015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:10:32.064015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:10:32.064015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:10:32.064015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:10:32.064015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:10:32.064015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:10:32.064015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:10:32.064015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:32.066502 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 252/492] 10.222.0.81 () {56 vars in 1093 bytes} [Mon Sep 19 11:10:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:10:32.067091 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3bdb2-a5a3-4164-95ef-5dcccdb8e429 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:32.077638 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38963ab-6a7d-4c0e-83e3-d8c8b5bae1e6 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:32.120416 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50adaf12-6777-4689-9098-aa4c5fd48315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:32.121046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adaf12-6777-4689-9098-aa4c5fd48315 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:32.121329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adaf12-6777-4689-9098-aa4c5fd48315 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:32.121598 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adaf12-6777-4689-9098-aa4c5fd48315 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:32.134651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50adaf12-6777-4689-9098-aa4c5fd48315 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:32.135131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:32.141970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-50adaf12-6777-4689-9098-aa4c5fd48315 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:32.141970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:32.141970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:32.141970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:32.141970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:32.141970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:32.141970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:32.141970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:32.141970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:32.141970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:32.141970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:32.141970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:32.141970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:32.141970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:32.141970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:32.141970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:32.142860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:32.142860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:32.142860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:32.142860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:32.142860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:32.142860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:32.142860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:32.142860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:32.142860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:32.142860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:32.142860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:32.142860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:32.142860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:32.142860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:32.142860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:32.142860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:32.143662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:32.143662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:32.143662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:32.143662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:32.143662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:32.143662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:32.143662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:32.143662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:32.143662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:32.143662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:32.143662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:32.143662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:32.147709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 241/493] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:10:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:32.148758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50adaf12-6777-4689-9098-aa4c5fd48315 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:32.176132 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3bdb2-a5a3-4164-95ef-5dcccdb8e429 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:32.208285 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:32.214585 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03679cec-4bef-4b9c-bf25-0064c2f8cf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:32.215508 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03679cec-4bef-4b9c-bf25-0064c2f8cf5a None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:32.215875 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03679cec-4bef-4b9c-bf25-0064c2f8cf5a None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:32.216147 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03679cec-4bef-4b9c-bf25-0064c2f8cf5a None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:32.216762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03679cec-4bef-4b9c-bf25-0064c2f8cf5a None admin] RBAC: Adding query filter params (name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:32.217071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03679cec-4bef-4b9c-bf25-0064c2f8cf5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:32.218598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03679cec-4bef-4b9c-bf25-0064c2f8cf5a None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:32.227482 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 253/494] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:10:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:32.229252 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d250250-9bb8-400f-9cc1-c9e208ba9059 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:32.229544 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d250250-9bb8-400f-9cc1-c9e208ba9059 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:32.229821 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d250250-9bb8-400f-9cc1-c9e208ba9059 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:32.231166 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 254/495] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:32.231799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d250250-9bb8-400f-9cc1-c9e208ba9059 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:32.259009 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50adaf12-6777-4689-9098-aa4c5fd48315 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:32.295604 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88cd00-08ce-427f-9ce4-eff2b4c6da92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:32.296236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88cd00-08ce-427f-9ce4-eff2b4c6da92 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:32.296544 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88cd00-08ce-427f-9ce4-eff2b4c6da92 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:32.296790 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88cd00-08ce-427f-9ce4-eff2b4c6da92 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:32.297524 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88cd00-08ce-427f-9ce4-eff2b4c6da92 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:32.297834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88cd00-08ce-427f-9ce4-eff2b4c6da92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:32.299262 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88cd00-08ce-427f-9ce4-eff2b4c6da92 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:32.340766 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 242/496] 10.222.0.81 () {56 vars in 1133 bytes} [Mon Sep 19 11:10:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:32.340766 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88cd00-08ce-427f-9ce4-eff2b4c6da92 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:32.348209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d250250-9bb8-400f-9cc1-c9e208ba9059 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:32.379297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:32.389081 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebf252-31c9-46e4-b582-d821eff6bcf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:32.389688 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebf252-31c9-46e4-b582-d821eff6bcf5 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:32.390065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebf252-31c9-46e4-b582-d821eff6bcf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:32.390236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebf252-31c9-46e4-b582-d821eff6bcf5 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:32.390898 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebf252-31c9-46e4-b582-d821eff6bcf5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:32.391200 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebf252-31c9-46e4-b582-d821eff6bcf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:32.392593 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebf252-31c9-46e4-b582-d821eff6bcf5 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:32.401036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 255/497] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:10:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:32.402218 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f974b1-9d34-49e6-a152-85e275c5251c None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:32.402498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f974b1-9d34-49e6-a152-85e275c5251c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:32.402772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f974b1-9d34-49e6-a152-85e275c5251c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:32.426726 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-21f974b1-9d34-49e6-a152-85e275c5251c None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:32.428412 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-21f974b1-9d34-49e6-a152-85e275c5251c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:32.453822 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88cd00-08ce-427f-9ce4-eff2b4c6da92 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:32.471578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:32.496487 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23407af0-9bbc-456d-99c3-8f49df95a921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:32.497599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23407af0-9bbc-456d-99c3-8f49df95a921 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:32.498080 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23407af0-9bbc-456d-99c3-8f49df95a921 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:32.498375 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 256/498] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:32.498767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23407af0-9bbc-456d-99c3-8f49df95a921 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:32.499222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f974b1-9d34-49e6-a152-85e275c5251c None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:32.519124 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23407af0-9bbc-456d-99c3-8f49df95a921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:32.521133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23407af0-9bbc-456d-99c3-8f49df95a921 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:32.541106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-23407af0-9bbc-456d-99c3-8f49df95a921 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:32.541106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:32.541106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:32.541106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:32.541106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:32.541106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:32.541106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:32.541106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:32.541106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:32.541106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:32.541106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:32.541106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:32.541106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:32.541106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:32.541106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:32.541106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:32.541998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:32.541998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:32.541998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:32.541998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:32.541998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:32.541998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:32.541998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:32.541998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:32.541998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:32.541998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:32.541998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:32.541998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:32.541998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:32.541998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:32.541998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:32.541998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:32.541998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:32.542869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:32.542869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:32.542869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:32.551918 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 243/499] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:32.551918 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23407af0-9bbc-456d-99c3-8f49df95a921 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:32.629635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f974b1-9d34-49e6-a152-85e275c5251c None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:32.655573 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:32.676147 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8bd33-67e2-48f8-b55b-f5b242c39d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:32.677254 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8bd33-67e2-48f8-b55b-f5b242c39d31 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:32.679805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8bd33-67e2-48f8-b55b-f5b242c39d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:32.679805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8bd33-67e2-48f8-b55b-f5b242c39d31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:32.685848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23407af0-9bbc-456d-99c3-8f49df95a921 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:32.728751 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf383a90-850c-402a-8764-5ebdb1b8cd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:32.729863 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf383a90-850c-402a-8764-5ebdb1b8cd09 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:32.730339 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf383a90-850c-402a-8764-5ebdb1b8cd09 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:32.730803 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf383a90-850c-402a-8764-5ebdb1b8cd09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:32.744827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf383a90-850c-402a-8764-5ebdb1b8cd09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:32.746637 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf383a90-850c-402a-8764-5ebdb1b8cd09 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:32.762186 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8bd33-67e2-48f8-b55b-f5b242c39d31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:32.762652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-bf383a90-850c-402a-8764-5ebdb1b8cd09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:32.762652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:32.762652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:32.762652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:32.762652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:32.762652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:32.762652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:32.762652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:32.762652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:32.762652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:32.762652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:32.762652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:32.762652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:32.762652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:32.762652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:32.762652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:32.763529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:32.763529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:32.763529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:32.763529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:32.763529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:32.763529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:32.763529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:32.763529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:32.763529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:32.763529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:32.763529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:32.763529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:32.763529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:32.763529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:32.763529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:32.763529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:32.763529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:32.764450 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:32.764450 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:32.764450 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:32.764613 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8bd33-67e2-48f8-b55b-f5b242c39d31 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:32.766769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 244/500] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:32.768069 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf383a90-850c-402a-8764-5ebdb1b8cd09 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:32.793387 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:32.869748 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 257/501] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:10:32.870592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8bd33-67e2-48f8-b55b-f5b242c39d31 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:32.875337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf383a90-850c-402a-8764-5ebdb1b8cd09 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:32.891328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:32.910134 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16ce81-be14-4ac1-9a7f-fa26a0d50e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:32.911128 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16ce81-be14-4ac1-9a7f-fa26a0d50e66 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:32.911410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16ce81-be14-4ac1-9a7f-fa26a0d50e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:32.911681 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16ce81-be14-4ac1-9a7f-fa26a0d50e66 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:32.912706 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16ce81-be14-4ac1-9a7f-fa26a0d50e66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:32.913016 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16ce81-be14-4ac1-9a7f-fa26a0d50e66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:32.914698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16ce81-be14-4ac1-9a7f-fa26a0d50e66 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:32.926121 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 245/502] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:32.927342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4718bb3-36e5-404c-bb4d-04d31747be31 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:32.927634 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4718bb3-36e5-404c-bb4d-04d31747be31 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:32.927934 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4718bb3-36e5-404c-bb4d-04d31747be31 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:32.929287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 246/503] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:32.929897 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4718bb3-36e5-404c-bb4d-04d31747be31 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:32.972081 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8bd33-67e2-48f8-b55b-f5b242c39d31 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:33.004154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755baf0-383a-4e85-8b4c-642215c768ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:33.004750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755baf0-383a-4e85-8b4c-642215c768ff None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:33.005021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755baf0-383a-4e85-8b4c-642215c768ff None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:33.005286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755baf0-383a-4e85-8b4c-642215c768ff None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:33.005912 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755baf0-383a-4e85-8b4c-642215c768ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:33.006209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755baf0-383a-4e85-8b4c-642215c768ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:33.007552 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755baf0-383a-4e85-8b4c-642215c768ff None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:33.008096 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:33.020251 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 258/504] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:33.021037 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755baf0-383a-4e85-8b4c-642215c768ff None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:33.032039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4718bb3-36e5-404c-bb4d-04d31747be31 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:33.065620 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f510b-0bf5-44e4-9250-71b8d0b3d055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:33.066231 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f510b-0bf5-44e4-9250-71b8d0b3d055 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:33.066506 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f510b-0bf5-44e4-9250-71b8d0b3d055 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:33.066774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f510b-0bf5-44e4-9250-71b8d0b3d055 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:33.076612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f510b-0bf5-44e4-9250-71b8d0b3d055 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:33.081787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-cf2f510b-0bf5-44e4-9250-71b8d0b3d055 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:33.081787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:33.081787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:33.081787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:33.081787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:33.081787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:33.081787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:33.081787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:33.081787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:33.081787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:33.081787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:33.081787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:33.081787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:33.081787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:33.081787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:33.081787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:33.082866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:33.082866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:33.082866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:33.082866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:33.082866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:33.082866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:33.082866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:33.082866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:33.082866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:33.082866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:33.082866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:33.082866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:33.082866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:33.082866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:33.082866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:33.082866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:33.083891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:33.083891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:33.083891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:33.083891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:33.083891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:33.083891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:33.083891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:33.083891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:33.083891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:33.083891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:33.083891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:33.083891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:33.083891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 247/505] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:10:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:33.083891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f510b-0bf5-44e4-9250-71b8d0b3d055 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:33.105532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:33.124552 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755baf0-383a-4e85-8b4c-642215c768ff None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:33.162699 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ad2ec-2d06-4746-8239-4a962010838a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:33.163239 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ad2ec-2d06-4746-8239-4a962010838a None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:33.163511 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ad2ec-2d06-4746-8239-4a962010838a None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:33.163811 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ad2ec-2d06-4746-8239-4a962010838a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:33.174548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ad2ec-2d06-4746-8239-4a962010838a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:33.176010 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ad2ec-2d06-4746-8239-4a962010838a None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:33.187281 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f510b-0bf5-44e4-9250-71b8d0b3d055 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:33.190203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-ec2ad2ec-2d06-4746-8239-4a962010838a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:33.190203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:33.190203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:33.190203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:33.190203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:33.190203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:33.190203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:33.190203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:33.190203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:33.190203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:33.190203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:33.190203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:33.190203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:33.190203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:33.190203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:33.190203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:33.191274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:33.191274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:33.191274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:33.191274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:33.191274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:33.191274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:33.191274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:33.191274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:33.191274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:33.191274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:33.191274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:33.191274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:33.191274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:33.191274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:33.191274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:33.191274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:33.191274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:33.192187 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:33.192187 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:33.192187 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:33.192187 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 259/506] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:33.192417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c73a7-fd03-4e15-a57e-752d70f5c8f2 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:33.192658 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c73a7-fd03-4e15-a57e-752d70f5c8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:33.192922 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c73a7-fd03-4e15-a57e-752d70f5c8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:33.211049 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-972c73a7-fd03-4e15-a57e-752d70f5c8f2 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:33.212790 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-972c73a7-fd03-4e15-a57e-752d70f5c8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:33.221227 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a94fd7-76d2-44b4-a7d0-5d5fd8c31ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:33.221847 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a94fd7-76d2-44b4-a7d0-5d5fd8c31ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:33.222125 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a94fd7-76d2-44b4-a7d0-5d5fd8c31ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:33.222393 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a94fd7-76d2-44b4-a7d0-5d5fd8c31ef1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:33.229503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a94fd7-76d2-44b4-a7d0-5d5fd8c31ef1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:33.230945 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a94fd7-76d2-44b4-a7d0-5d5fd8c31ef1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:33.232697 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:33.244719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-65a94fd7-76d2-44b4-a7d0-5d5fd8c31ef1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 19 11:10:33.244719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:33.244719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:33.244719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:33.244719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:33.244719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:33.244719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:33.244719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:33.244719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:33.244719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:33.244719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:33.244719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:33.244719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:10:33.244719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:10:33.244719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:10:33.244719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:10:33.245873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:33.245873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:33.245873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:10:33.245873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:33.245873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:10:33.245873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:33.245873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:33.245873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:33.245873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:33.245873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:33.245873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:33.245873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:33.245873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:33.245873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:33.245873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:33.245873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:33.245873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:33.245873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:33.246767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:33.246767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:33.246767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:10:33.246767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:10:33.246767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:10:33.246767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:10:33.246767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:10:33.246767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:10:33.246767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 19 11:10:33.246767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:33.246767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 248/507] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:33.246767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a94fd7-76d2-44b4-a7d0-5d5fd8c31ef1 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:33.284961 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 260/508] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:33.286200 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c73a7-fd03-4e15-a57e-752d70f5c8f2 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:33.354749 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a94fd7-76d2-44b4-a7d0-5d5fd8c31ef1 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:33.360074 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:33.388447 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bcd6b5-0cd4-4077-9d28-8c5966b3141c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:33.389063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcd6b5-0cd4-4077-9d28-8c5966b3141c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:33.389345 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcd6b5-0cd4-4077-9d28-8c5966b3141c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:33.389617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcd6b5-0cd4-4077-9d28-8c5966b3141c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:33.398185 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bcd6b5-0cd4-4077-9d28-8c5966b3141c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:33.400657 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bcd6b5-0cd4-4077-9d28-8c5966b3141c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:33.402388 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c73a7-fd03-4e15-a57e-752d70f5c8f2 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:33.411892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-21bcd6b5-0cd4-4077-9d28-8c5966b3141c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:10:33.411892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:33.411892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:33.411892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:33.411892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:33.411892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:33.411892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:33.411892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:33.411892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:33.411892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:33.411892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:33.411892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:33.411892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:10:33.411892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:10:33.411892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:10:33.411892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:10:33.417787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:33.417787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:33.417787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:10:33.417787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:33.417787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:10:33.417787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:33.417787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:33.417787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:33.417787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:33.417787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:33.417787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:33.417787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:33.417787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:33.417787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:33.417787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:33.417787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:33.417787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:33.417787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:33.418942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:33.418942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:33.418942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:10:33.418942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:10:33.418942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:10:33.418942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:10:33.418942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:10:33.418942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:10:33.418942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:10:33.418942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:33.418942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 249/509] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:33.418942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bcd6b5-0cd4-4077-9d28-8c5966b3141c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:33.445254 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b379e-ca3b-4ff2-81d9-cf81f86a9290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:33.446225 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b379e-ca3b-4ff2-81d9-cf81f86a9290 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:33.447170 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b379e-ca3b-4ff2-81d9-cf81f86a9290 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:33.447481 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b379e-ca3b-4ff2-81d9-cf81f86a9290 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:33.448269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b379e-ca3b-4ff2-81d9-cf81f86a9290 None admin] RBAC: Adding query filter params (name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:33.448663 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b379e-ca3b-4ff2-81d9-cf81f86a9290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:33.450104 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b379e-ca3b-4ff2-81d9-cf81f86a9290 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:33.458507 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 261/510] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:10:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:33.463688 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b379e-ca3b-4ff2-81d9-cf81f86a9290 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:33.476111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:33.520090 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bcd6b5-0cd4-4077-9d28-8c5966b3141c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:33.555951 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d007e-aa2c-4506-be38-8f44c6e98bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:33.557236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d007e-aa2c-4506-be38-8f44c6e98bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:33.557457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d007e-aa2c-4506-be38-8f44c6e98bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:33.557729 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d007e-aa2c-4506-be38-8f44c6e98bb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:33.558361 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d007e-aa2c-4506-be38-8f44c6e98bb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:33.558658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d007e-aa2c-4506-be38-8f44c6e98bb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:33.560041 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d007e-aa2c-4506-be38-8f44c6e98bb7 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:33.570411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b379e-ca3b-4ff2-81d9-cf81f86a9290 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:33.571380 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 250/511] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:33.572214 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d007e-aa2c-4506-be38-8f44c6e98bb7 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:33.593620 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:33.605194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f83b7-dec5-499f-8bec-9d4efc0f317b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:33.605793 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f83b7-dec5-499f-8bec-9d4efc0f317b None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:33.606066 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f83b7-dec5-499f-8bec-9d4efc0f317b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:33.606333 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f83b7-dec5-499f-8bec-9d4efc0f317b None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:33.607057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f83b7-dec5-499f-8bec-9d4efc0f317b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:33.607361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f83b7-dec5-499f-8bec-9d4efc0f317b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:33.608802 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f83b7-dec5-499f-8bec-9d4efc0f317b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:33.644188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 262/512] 10.222.0.81 () {56 vars in 1137 bytes} [Mon Sep 19 11:10:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:33.644188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f83b7-dec5-499f-8bec-9d4efc0f317b None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:33.674888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d007e-aa2c-4506-be38-8f44c6e98bb7 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:33.692108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:33.710075 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bf79e-234c-414c-b42f-98d433ca04d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:33.710671 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bf79e-234c-414c-b42f-98d433ca04d7 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:33.710950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bf79e-234c-414c-b42f-98d433ca04d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:33.711225 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bf79e-234c-414c-b42f-98d433ca04d7 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:33.711975 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bf79e-234c-414c-b42f-98d433ca04d7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:33.712332 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bf79e-234c-414c-b42f-98d433ca04d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:33.713753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bf79e-234c-414c-b42f-98d433ca04d7 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:33.730730 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 251/513] 10.222.0.81 () {56 vars in 1137 bytes} [Mon Sep 19 11:10:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:33.731518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bf79e-234c-414c-b42f-98d433ca04d7 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:33.745164 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f83b7-dec5-499f-8bec-9d4efc0f317b None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:33.779041 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002d275-5256-49ad-935f-d2a06d1eeaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:33.779662 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002d275-5256-49ad-935f-d2a06d1eeaad None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:33.779971 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002d275-5256-49ad-935f-d2a06d1eeaad None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:33.780246 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002d275-5256-49ad-935f-d2a06d1eeaad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:33.790225 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002d275-5256-49ad-935f-d2a06d1eeaad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:33.791831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002d275-5256-49ad-935f-d2a06d1eeaad None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:33.806852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-1002d275-5256-49ad-935f-d2a06d1eeaad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:33.806852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:33.806852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:33.806852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:33.806852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:33.806852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:33.806852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:33.806852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:33.806852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:33.806852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:33.806852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:33.806852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:33.806852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:33.806852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:33.806852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:33.806852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:33.807839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:33.807839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:33.807839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:33.807839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:33.807839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:33.807839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:33.807839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:33.807839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:33.807839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:33.807839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:33.807839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:33.807839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:33.807839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:33.807839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:33.807839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:33.807839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:33.807839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:33.808693 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:33.808693 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:33.808693 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:33.808693 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 263/514] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:33.808693 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002d275-5256-49ad-935f-d2a06d1eeaad None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:33.828328 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bf79e-234c-414c-b42f-98d433ca04d7 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:33.842274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:33.862228 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4013d62-735b-4ce1-9868-8320f73d2cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:33.862849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4013d62-735b-4ce1-9868-8320f73d2cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:33.863134 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4013d62-735b-4ce1-9868-8320f73d2cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:33.863402 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4013d62-735b-4ce1-9868-8320f73d2cb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:33.873709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4013d62-735b-4ce1-9868-8320f73d2cb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:33.875157 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4013d62-735b-4ce1-9868-8320f73d2cb6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:33.889355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-d4013d62-735b-4ce1-9868-8320f73d2cb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:33.889355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:33.889355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:33.889355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:33.889355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:33.889355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:33.889355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:33.889355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:33.889355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:33.889355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:33.889355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:33.889355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:33.889355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:33.889355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:33.889355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:33.889355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:33.890236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:33.890236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:33.890236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:33.890236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:33.890236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:33.890236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:33.890236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:33.890236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:33.890236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:33.890236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:33.890236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:33.890236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:33.890236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:33.890236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:33.890236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:33.890236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:33.890236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:33.891075 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:33.891075 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:33.891075 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:33.891075 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 252/515] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:33.891528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4013d62-735b-4ce1-9868-8320f73d2cb6 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:33.911116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002d275-5256-49ad-935f-d2a06d1eeaad None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:33.918771 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:33.944857 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ceabe-76ab-4084-ab21-35f4a3fa43f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:33.945401 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ceabe-76ab-4084-ab21-35f4a3fa43f2 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:33.945675 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ceabe-76ab-4084-ab21-35f4a3fa43f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:33.945936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ceabe-76ab-4084-ab21-35f4a3fa43f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:33.952365 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ceabe-76ab-4084-ab21-35f4a3fa43f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:33.954037 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ceabe-76ab-4084-ab21-35f4a3fa43f2 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:33.966118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-ec0ceabe-76ab-4084-ab21-35f4a3fa43f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:33.966118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:33.966118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:33.966118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:33.966118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:33.966118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:33.966118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:33.966118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:33.966118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:33.966118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:33.966118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:33.966118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:33.966118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:10:33.966118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:10:33.966118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:10:33.966955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:10:33.966955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:33.966955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:33.966955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:33.966955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:33.966955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:33.966955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:33.966955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:33.966955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:33.966955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:33.966955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:33.966955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:33.966955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:33.966955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:33.966955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:33.966955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:33.966955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:33.967838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:10:33.967838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:10:33.967838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:10:33.967838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:10:33.967838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:10:33.967838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:10:33.967838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:33.967838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:33.967838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 264/516] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:33.968317 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ceabe-76ab-4084-ab21-35f4a3fa43f2 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:33.995216 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4013d62-735b-4ce1-9868-8320f73d2cb6 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:34.028009 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf9cc9-ecc8-41a6-b539-395cbcf890d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:34.028603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf9cc9-ecc8-41a6-b539-395cbcf890d9 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:34.028879 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf9cc9-ecc8-41a6-b539-395cbcf890d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:34.029150 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf9cc9-ecc8-41a6-b539-395cbcf890d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:34.063256 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:34.069775 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ceabe-76ab-4084-ab21-35f4a3fa43f2 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:34.097044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf9cc9-ecc8-41a6-b539-395cbcf890d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:34.098525 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf9cc9-ecc8-41a6-b539-395cbcf890d9 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:34.102940 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387690c-4816-462e-860f-3383635a9ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:34.104080 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387690c-4816-462e-860f-3383635a9ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:34.104361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387690c-4816-462e-860f-3383635a9ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:34.104632 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387690c-4816-462e-860f-3383635a9ec9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:34.124787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:34.169155 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387690c-4816-462e-860f-3383635a9ec9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:34.170590 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387690c-4816-462e-860f-3383635a9ec9 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:34.195595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 253/517] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:10:34.197267 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf9cc9-ecc8-41a6-b539-395cbcf890d9 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:34.265132 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:34.272901 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 265/518] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:10:34.273695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387690c-4816-462e-860f-3383635a9ec9 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:34.295223 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf9cc9-ecc8-41a6-b539-395cbcf890d9 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:34.320449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:34.331462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f86ec6-6f1a-4484-9de1-a2b1a6feb35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:34.332103 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f86ec6-6f1a-4484-9de1-a2b1a6feb35e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:34.332382 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f86ec6-6f1a-4484-9de1-a2b1a6feb35e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:34.332657 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f86ec6-6f1a-4484-9de1-a2b1a6feb35e None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:34.333298 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f86ec6-6f1a-4484-9de1-a2b1a6feb35e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:34.333601 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f86ec6-6f1a-4484-9de1-a2b1a6feb35e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:34.334973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f86ec6-6f1a-4484-9de1-a2b1a6feb35e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:34.346785 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 254/519] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:34.347578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f86ec6-6f1a-4484-9de1-a2b1a6feb35e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:34.370664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387690c-4816-462e-860f-3383635a9ec9 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:34.405275 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7be21-ad5c-47df-8c5e-e25d2cdbb0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:34.405867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7be21-ad5c-47df-8c5e-e25d2cdbb0ab None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:34.406144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7be21-ad5c-47df-8c5e-e25d2cdbb0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:34.406410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7be21-ad5c-47df-8c5e-e25d2cdbb0ab None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:34.407039 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7be21-ad5c-47df-8c5e-e25d2cdbb0ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:34.407339 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7be21-ad5c-47df-8c5e-e25d2cdbb0ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:34.408757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7be21-ad5c-47df-8c5e-e25d2cdbb0ab None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:34.419562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 266/520] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:34.420371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7be21-ad5c-47df-8c5e-e25d2cdbb0ab None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:34.449935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f86ec6-6f1a-4484-9de1-a2b1a6feb35e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:34.473274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:34.483664 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6082221-55fe-4e46-b7b3-7eb197022a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:34.484299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6082221-55fe-4e46-b7b3-7eb197022a51 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:34.484670 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6082221-55fe-4e46-b7b3-7eb197022a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:34.484950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6082221-55fe-4e46-b7b3-7eb197022a51 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:34.485643 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6082221-55fe-4e46-b7b3-7eb197022a51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:34.485951 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6082221-55fe-4e46-b7b3-7eb197022a51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:34.487414 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6082221-55fe-4e46-b7b3-7eb197022a51 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:34.504602 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 255/521] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:10:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:34.505406 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6082221-55fe-4e46-b7b3-7eb197022a51 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:34.522102 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7be21-ad5c-47df-8c5e-e25d2cdbb0ab None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:34.556606 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acede25d-bdcb-41cc-afcf-1bba11266bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:34.557227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acede25d-bdcb-41cc-afcf-1bba11266bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:34.557501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acede25d-bdcb-41cc-afcf-1bba11266bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:34.557767 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acede25d-bdcb-41cc-afcf-1bba11266bb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:34.562679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:34.568327 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acede25d-bdcb-41cc-afcf-1bba11266bb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:34.569662 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acede25d-bdcb-41cc-afcf-1bba11266bb2 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:34.585738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-acede25d-bdcb-41cc-afcf-1bba11266bb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:34.585738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:34.585738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:34.585738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:34.585738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:34.585738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:34.585738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:34.585738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:34.585738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:34.585738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:34.585738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:34.585738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:34.585738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:34.585738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:34.585738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:34.585738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:34.586704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:34.586704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:34.586704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:34.586704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:34.586704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:34.586704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:34.586704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:34.586704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:34.586704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:34.586704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:34.586704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:34.586704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:34.586704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:34.586704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:34.586704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:34.586704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:34.586704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:34.587587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:34.587587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:34.587587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:34.587587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 267/522] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:34.587587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acede25d-bdcb-41cc-afcf-1bba11266bb2 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:34.613477 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6082221-55fe-4e46-b7b3-7eb197022a51 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:34.647591 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3a195-506b-4686-87b5-4996eb4deea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:34.648243 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3a195-506b-4686-87b5-4996eb4deea0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:34.648525 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3a195-506b-4686-87b5-4996eb4deea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:34.648798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3a195-506b-4686-87b5-4996eb4deea0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:34.658748 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a3a195-506b-4686-87b5-4996eb4deea0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:34.660253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a3a195-506b-4686-87b5-4996eb4deea0 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:34.677669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-f3a3a195-506b-4686-87b5-4996eb4deea0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:34.677669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:34.677669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:34.677669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:34.677669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:34.677669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:34.677669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:34.677669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:34.677669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:34.677669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:34.677669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:34.677669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:34.677669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:34.677669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:34.677669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:34.677669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:34.678741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:34.678741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:34.678741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:34.678741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:34.678741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:34.678741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:34.678741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:34.678741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:34.678741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:34.678741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:34.678741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:34.678741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:34.678741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:34.678741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:34.678741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:34.678741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:34.678741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:34.679609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:34.679609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:34.679609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:34.679609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 256/523] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:34.679609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3a195-506b-4686-87b5-4996eb4deea0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:34.692954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acede25d-bdcb-41cc-afcf-1bba11266bb2 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:34.700652 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:34.726939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fa25f-b7ed-4a8e-a23f-2cf9354104d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:34.727552 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fa25f-b7ed-4a8e-a23f-2cf9354104d8 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:34.727906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fa25f-b7ed-4a8e-a23f-2cf9354104d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:34.728165 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fa25f-b7ed-4a8e-a23f-2cf9354104d8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:34.735270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fa25f-b7ed-4a8e-a23f-2cf9354104d8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:34.736730 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fa25f-b7ed-4a8e-a23f-2cf9354104d8 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:34.748007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-a89fa25f-b7ed-4a8e-a23f-2cf9354104d8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 11:10:34.748007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:34.748007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:34.748007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:34.748007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:34.748007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:34.748007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:34.748007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:34.748007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:34.748007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:34.748007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:34.748007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:34.748007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:10:34.748007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:10:34.748007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:10:34.748007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:10:34.749044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:34.749044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:34.749044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:10:34.749044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:34.749044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:10:34.749044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:34.749044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:34.749044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:34.749044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:34.749044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:34.749044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:34.749044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:34.749044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:34.749044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:34.749044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:34.749044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:34.749044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:34.749044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:34.749936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:34.749936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:34.749936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:10:34.749936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:10:34.749936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:10:34.749936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:10:34.749936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:10:34.749936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:10:34.749936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 11:10:34.749936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:34.749936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 268/524] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:10:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:34.749936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fa25f-b7ed-4a8e-a23f-2cf9354104d8 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:34.782126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:34.786302 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3a195-506b-4686-87b5-4996eb4deea0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:34.821243 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2ec2e-55fd-434f-809d-dff688189768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:34.822497 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2ec2e-55fd-434f-809d-dff688189768 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:34.822890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2ec2e-55fd-434f-809d-dff688189768 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:34.823285 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2ec2e-55fd-434f-809d-dff688189768 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:34.824051 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2ec2e-55fd-434f-809d-dff688189768 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:34.825491 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2ec2e-55fd-434f-809d-dff688189768 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:34.846191 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 257/525] 10.222.0.81 () {60 vars in 1132 bytes} [Mon Sep 19 11:10:34 2022] POST /identity/v3/users => generated 247 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:10:34.846959 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2ec2e-55fd-434f-809d-dff688189768 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:34.847459 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fa25f-b7ed-4a8e-a23f-2cf9354104d8 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:34.886670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4d97f-36c3-4626-848d-af630cbebb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:34.887366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4d97f-36c3-4626-848d-af630cbebb68 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:34.887654 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4d97f-36c3-4626-848d-af630cbebb68 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:34.887950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4d97f-36c3-4626-848d-af630cbebb68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:34.888692 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4d97f-36c3-4626-848d-af630cbebb68 None admin] RBAC: Adding query filter params (role.id=af9d39307d714527b14b2045ab466e7a, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=5abd70a18ccd407bafe3b61c8870986e) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:34.889028 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4d97f-36c3-4626-848d-af630cbebb68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:34.890532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4d97f-36c3-4626-848d-af630cbebb68 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:34.897174 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 269/526] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:10:34 2022] GET /identity/v3/role_assignments?user.id=5abd70a18ccd407bafe3b61c8870986e&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=af9d39307d714527b14b2045ab466e7a => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:34.897174 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4d97f-36c3-4626-848d-af630cbebb68 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:34.947446 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:34.996748 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2ec2e-55fd-434f-809d-dff688189768 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:35.039926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc3359-efb3-495c-a806-5082fbcb79a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:35.040728 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc3359-efb3-495c-a806-5082fbcb79a9 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:35.041198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc3359-efb3-495c-a806-5082fbcb79a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:35.041544 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc3359-efb3-495c-a806-5082fbcb79a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:35.042398 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc3359-efb3-495c-a806-5082fbcb79a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:35.042772 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc3359-efb3-495c-a806-5082fbcb79a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:35.044434 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc3359-efb3-495c-a806-5082fbcb79a9 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:35.050355 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4d97f-36c3-4626-848d-af630cbebb68 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:35.061471 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 258/527] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:35.062466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc3359-efb3-495c-a806-5082fbcb79a9 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:35.086553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:35.098959 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eaac11-8455-4506-9d91-de232731bf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:35.100501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaac11-8455-4506-9d91-de232731bf4e None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:35.100501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaac11-8455-4506-9d91-de232731bf4e None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:35.100501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaac11-8455-4506-9d91-de232731bf4e None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:35.101229 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eaac11-8455-4506-9d91-de232731bf4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:35.101594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eaac11-8455-4506-9d91-de232731bf4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:35.103217 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eaac11-8455-4506-9d91-de232731bf4e None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:35.118645 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 270/528] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:35.119990 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eaac11-8455-4506-9d91-de232731bf4e None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:35.202474 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc3359-efb3-495c-a806-5082fbcb79a9 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:35.222338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:35.238323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eaac11-8455-4506-9d91-de232731bf4e None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:35.254159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3031947-1193-4489-a812-cb24c92fd52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:35.255272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3031947-1193-4489-a812-cb24c92fd52d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:35.255738 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3031947-1193-4489-a812-cb24c92fd52d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:35.256436 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3031947-1193-4489-a812-cb24c92fd52d None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:35.257750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3031947-1193-4489-a812-cb24c92fd52d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:35.258245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3031947-1193-4489-a812-cb24c92fd52d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:35.260493 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3031947-1193-4489-a812-cb24c92fd52d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:35.279274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905db54-89c0-4b00-aa15-e28b425556b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:35.280044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905db54-89c0-4b00-aa15-e28b425556b3 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:35.280393 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905db54-89c0-4b00-aa15-e28b425556b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:35.280759 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905db54-89c0-4b00-aa15-e28b425556b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:35.291385 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 259/529] 10.222.0.81 () {56 vars in 1139 bytes} [Mon Sep 19 11:10:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:35.291385 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3031947-1193-4489-a812-cb24c92fd52d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:35.292722 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905db54-89c0-4b00-aa15-e28b425556b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:35.294175 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905db54-89c0-4b00-aa15-e28b425556b3 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:35.307092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-d905db54-89c0-4b00-aa15-e28b425556b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:35.307092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:35.307092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:35.307092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:35.307092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:35.307092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:35.307092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:35.307092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:35.307092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:35.307092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:35.307092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:35.307092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:35.307092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:10:35.307092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:10:35.307092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:10:35.308020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:10:35.308020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:35.308020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:35.308020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:35.308020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:35.308020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:35.308020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:35.308020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:35.308020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:35.308020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:35.308020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:35.308020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:35.308020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:35.308020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:35.308020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:35.308020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:35.308020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:35.309500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:10:35.309500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:10:35.309500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:10:35.309500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:10:35.309500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:10:35.309500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:10:35.309500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:35.309500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:35.309500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 271/530] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:35.309500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905db54-89c0-4b00-aa15-e28b425556b3 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:35.400834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:35.428124 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905db54-89c0-4b00-aa15-e28b425556b3 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:35.428648 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3031947-1193-4489-a812-cb24c92fd52d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:35.465717 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6648c8-2b81-4c25-a15e-7af1895a469c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:35.466441 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6648c8-2b81-4c25-a15e-7af1895a469c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:35.466721 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6648c8-2b81-4c25-a15e-7af1895a469c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:35.466993 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6648c8-2b81-4c25-a15e-7af1895a469c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:35.473091 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73a83b-d149-47bb-8203-9094d715d69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:35.474043 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6648c8-2b81-4c25-a15e-7af1895a469c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:35.474255 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73a83b-d149-47bb-8203-9094d715d69b None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:35.474744 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73a83b-d149-47bb-8203-9094d715d69b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:35.475226 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73a83b-d149-47bb-8203-9094d715d69b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:35.475484 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6648c8-2b81-4c25-a15e-7af1895a469c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:35.488166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-fc6648c8-2b81-4c25-a15e-7af1895a469c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:35.488166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:35.488166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:35.488166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:35.488166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:35.488166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:35.488166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:35.488166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:35.488166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:35.488166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:35.488166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:35.488166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:35.488166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:10:35.488166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:10:35.488166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:10:35.489048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:10:35.489048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:35.489048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:35.489048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:35.489048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:35.489048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:35.489048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:35.489048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:35.489048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:35.489048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:35.489048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:35.489048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:35.489048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:35.489048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:35.489048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:35.489048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:35.489048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:35.490644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:10:35.490644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:10:35.490644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:10:35.490644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:10:35.490644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:10:35.490644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:10:35.490644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:35.490644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:35.490644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 260/531] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:35.490644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6648c8-2b81-4c25-a15e-7af1895a469c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:35.501364 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:35.559497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73a83b-d149-47bb-8203-9094d715d69b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:35.560889 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73a83b-d149-47bb-8203-9094d715d69b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:35.632087 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6648c8-2b81-4c25-a15e-7af1895a469c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:35.673955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 272/532] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:10:35.675474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73a83b-d149-47bb-8203-9094d715d69b None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:35.683999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3b897-dd51-4bac-bb08-6c52c656d3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:35.684870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3b897-dd51-4bac-bb08-6c52c656d3ef None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:35.685330 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3b897-dd51-4bac-bb08-6c52c656d3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:35.685778 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3b897-dd51-4bac-bb08-6c52c656d3ef None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:35.687189 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad3b897-dd51-4bac-bb08-6c52c656d3ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:35.687707 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad3b897-dd51-4bac-bb08-6c52c656d3ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:35.689853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad3b897-dd51-4bac-bb08-6c52c656d3ef None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:35.692575 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:35.709242 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 261/533] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:10:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:35.710570 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3b897-dd51-4bac-bb08-6c52c656d3ef None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:35.732039 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:35.783353 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73a83b-d149-47bb-8203-9094d715d69b None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:35.819798 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caaceb3-9d6d-4088-93ee-ca11b00a6d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:35.820461 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caaceb3-9d6d-4088-93ee-ca11b00a6d76 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:35.820829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caaceb3-9d6d-4088-93ee-ca11b00a6d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:35.821171 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caaceb3-9d6d-4088-93ee-ca11b00a6d76 None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:35.822500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caaceb3-9d6d-4088-93ee-ca11b00a6d76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:35.822962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caaceb3-9d6d-4088-93ee-ca11b00a6d76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:35.824614 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caaceb3-9d6d-4088-93ee-ca11b00a6d76 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:35.827826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3b897-dd51-4bac-bb08-6c52c656d3ef None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:35.845522 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 273/534] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:10:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:35.846599 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caaceb3-9d6d-4088-93ee-ca11b00a6d76 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:35.869632 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f8844-c39e-4f1c-9092-59549fcb5261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:35.870564 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f8844-c39e-4f1c-9092-59549fcb5261 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:35.871053 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f8844-c39e-4f1c-9092-59549fcb5261 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:35.871549 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f8844-c39e-4f1c-9092-59549fcb5261 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:35.883145 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f8844-c39e-4f1c-9092-59549fcb5261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:35.885401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f8844-c39e-4f1c-9092-59549fcb5261 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:35.901894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-a45f8844-c39e-4f1c-9092-59549fcb5261 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:35.901894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:35.901894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:35.901894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:35.901894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:35.901894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:35.901894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:35.901894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:35.901894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:35.901894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:35.901894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:35.901894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:35.901894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:35.901894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:35.901894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:35.901894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:35.902804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:35.902804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:35.902804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:35.902804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:35.902804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:35.902804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:35.902804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:35.902804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:35.902804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:35.902804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:35.902804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:35.902804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:35.902804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:35.902804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:35.902804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:35.902804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:35.902804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:35.903640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:35.903640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:35.903640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:35.910261 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 262/535] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:35.911392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f8844-c39e-4f1c-9092-59549fcb5261 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:35.951494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caaceb3-9d6d-4088-93ee-ca11b00a6d76 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:35.962103 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:35.963143 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:35.988622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28dcd35-0e5a-4c98-ab10-74c76dbca087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:35.989311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dcd35-0e5a-4c98-ab10-74c76dbca087 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:35.989592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dcd35-0e5a-4c98-ab10-74c76dbca087 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:35.989863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dcd35-0e5a-4c98-ab10-74c76dbca087 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:35.990570 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28dcd35-0e5a-4c98-ab10-74c76dbca087 None admin] RBAC: Adding query filter params (role.id=af9d39307d714527b14b2045ab466e7a, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=88670b0e1a0f460d8f74917c92db9b9a) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:35.990879 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28dcd35-0e5a-4c98-ab10-74c76dbca087 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:35.992724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28dcd35-0e5a-4c98-ab10-74c76dbca087 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:35.999474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 274/536] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:10:35 2022] GET /identity/v3/role_assignments?user.id=88670b0e1a0f460d8f74917c92db9b9a&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=af9d39307d714527b14b2045ab466e7a => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:36.000476 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28dcd35-0e5a-4c98-ab10-74c76dbca087 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:36.029556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f8844-c39e-4f1c-9092-59549fcb5261 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:36.072248 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94298e96-a6ac-4f84-aaac-d5ab53c06063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:36.073759 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94298e96-a6ac-4f84-aaac-d5ab53c06063 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:36.074356 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94298e96-a6ac-4f84-aaac-d5ab53c06063 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:36.074835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94298e96-a6ac-4f84-aaac-d5ab53c06063 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:36.075802 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94298e96-a6ac-4f84-aaac-d5ab53c06063 None admin] RBAC: Adding query filter params (role.id=af9d39307d714527b14b2045ab466e7a, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=f9f494df37254102b339df35a151e075) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:36.076336 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94298e96-a6ac-4f84-aaac-d5ab53c06063 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:36.078234 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94298e96-a6ac-4f84-aaac-d5ab53c06063 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:36.085316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 263/537] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:10:35 2022] GET /identity/v3/role_assignments?user.id=f9f494df37254102b339df35a151e075&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=af9d39307d714527b14b2045ab466e7a => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:36.116367 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28dcd35-0e5a-4c98-ab10-74c76dbca087 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:36.163676 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627d3ad-c700-4ee5-8f10-9a811b6aa60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:36.164325 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627d3ad-c700-4ee5-8f10-9a811b6aa60c None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:36.164603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627d3ad-c700-4ee5-8f10-9a811b6aa60c None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:36.164874 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627d3ad-c700-4ee5-8f10-9a811b6aa60c None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:36.165595 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627d3ad-c700-4ee5-8f10-9a811b6aa60c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:36.165896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627d3ad-c700-4ee5-8f10-9a811b6aa60c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:36.167328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627d3ad-c700-4ee5-8f10-9a811b6aa60c None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:36.181941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 275/538] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:36.187903 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94298e96-a6ac-4f84-aaac-d5ab53c06063 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:36.321013 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94298e96-a6ac-4f84-aaac-d5ab53c06063 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:36.333867 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:36.361106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fa0e9-2036-4f43-bca8-ab3ba207ff34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:36.362078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fa0e9-2036-4f43-bca8-ab3ba207ff34 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:36.362561 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fa0e9-2036-4f43-bca8-ab3ba207ff34 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:36.363036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fa0e9-2036-4f43-bca8-ab3ba207ff34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:36.370207 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2fa0e9-2036-4f43-bca8-ab3ba207ff34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:36.371935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2fa0e9-2036-4f43-bca8-ab3ba207ff34 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:36.383593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-cf2fa0e9-2036-4f43-bca8-ab3ba207ff34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:36.383593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:36.383593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:36.383593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:36.383593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:36.383593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:36.383593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:36.383593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:36.383593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:36.383593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:36.383593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:36.383593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:36.383593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:10:36.383593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:10:36.383593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:10:36.384520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:10:36.384520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:36.384520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:36.384520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:36.384520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:36.384520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:36.384520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:36.384520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:36.384520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:36.384520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:36.384520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:36.384520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:36.384520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:36.384520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:36.384520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:36.384520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:36.384520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:36.385384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:10:36.385384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:10:36.385384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:10:36.385384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:10:36.385384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:10:36.385384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:10:36.385384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:36.385384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:36.391998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 264/539] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:36.394413 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627d3ad-c700-4ee5-8f10-9a811b6aa60c None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:36.438878 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:36.526392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627d3ad-c700-4ee5-8f10-9a811b6aa60c None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:36.572258 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c1a98-b1ec-47a8-8d4c-102c2655e52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:36.572872 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c1a98-b1ec-47a8-8d4c-102c2655e52f None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:36.573148 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c1a98-b1ec-47a8-8d4c-102c2655e52f None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:36.573418 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c1a98-b1ec-47a8-8d4c-102c2655e52f None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:36.574110 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c1a98-b1ec-47a8-8d4c-102c2655e52f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:36.574414 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c1a98-b1ec-47a8-8d4c-102c2655e52f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:36.575873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c1a98-b1ec-47a8-8d4c-102c2655e52f None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:36.593174 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 276/540] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:10:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:36.599159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fa0e9-2036-4f43-bca8-ab3ba207ff34 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:36.724812 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fa0e9-2036-4f43-bca8-ab3ba207ff34 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:36.762882 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1ea3d-ea44-4a50-a327-38a12f71f976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:36.763490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1ea3d-ea44-4a50-a327-38a12f71f976 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:36.763786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1ea3d-ea44-4a50-a327-38a12f71f976 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:36.764079 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1ea3d-ea44-4a50-a327-38a12f71f976 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:36.764812 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1ea3d-ea44-4a50-a327-38a12f71f976 None admin] RBAC: Adding query filter params (role.id=af9d39307d714527b14b2045ab466e7a, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=65de2e5f67dd4bb6a20cf3934afc8abd) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:36.765186 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1ea3d-ea44-4a50-a327-38a12f71f976 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:36.766682 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1ea3d-ea44-4a50-a327-38a12f71f976 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:36.779937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 265/541] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:10:36 2022] GET /identity/v3/role_assignments?user.id=65de2e5f67dd4bb6a20cf3934afc8abd&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=af9d39307d714527b14b2045ab466e7a => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:37.245286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a681f9-1a38-4ffb-9393-ccfba4335688 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:37.246024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a681f9-1a38-4ffb-9393-ccfba4335688 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:37.246515 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a681f9-1a38-4ffb-9393-ccfba4335688 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:37.248496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 277/542] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:37.253570 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94a466-fb99-4d9b-b053-08aac58ecd06 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:37.254087 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94a466-fb99-4d9b-b053-08aac58ecd06 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:37.254573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94a466-fb99-4d9b-b053-08aac58ecd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:37.280405 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8e94a466-fb99-4d9b-b053-08aac58ecd06 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:37.282038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8e94a466-fb99-4d9b-b053-08aac58ecd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:37.320763 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:37.376988 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 266/543] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:37.431555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edcf3f-1ee0-4d67-b1c3-010772d6ea37 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:37.432222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edcf3f-1ee0-4d67-b1c3-010772d6ea37 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:37.432698 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edcf3f-1ee0-4d67-b1c3-010772d6ea37 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:37.434240 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 278/544] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:37.439524 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b1daf-3d36-4780-b62a-2b7e63afc265 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:37.440067 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b1daf-3d36-4780-b62a-2b7e63afc265 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:37.440546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b1daf-3d36-4780-b62a-2b7e63afc265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:37.464710 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-674b1daf-3d36-4780-b62a-2b7e63afc265 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:37.466223 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-674b1daf-3d36-4780-b62a-2b7e63afc265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:37.558403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 267/545] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:37.736234 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edcf3f-1ee0-4d67-b1c3-010772d6ea37 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:37.839786 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:37.857554 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edcf3f-1ee0-4d67-b1c3-010772d6ea37 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:37.898795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29363d-a7af-44d2-bd39-7e0a44ae1741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:37.899432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29363d-a7af-44d2-bd39-7e0a44ae1741 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:37.900301 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29363d-a7af-44d2-bd39-7e0a44ae1741 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:37.900578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29363d-a7af-44d2-bd39-7e0a44ae1741 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:37.916109 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be29363d-a7af-44d2-bd39-7e0a44ae1741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:37.920905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-be29363d-a7af-44d2-bd39-7e0a44ae1741 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:37.920905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:37.920905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:37.920905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:37.920905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:37.920905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:37.920905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:37.920905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:37.920905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:37.920905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:37.920905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:37.920905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:37.920905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:37.920905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:37.920905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:37.920905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:37.921799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:37.921799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:37.921799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:37.921799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:37.921799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:37.921799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:37.921799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:37.921799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:37.921799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:37.921799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:37.921799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:37.921799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:37.921799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:37.921799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:37.921799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:37.921799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:37.923192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:37.923192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:37.923192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:37.923192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:37.923192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:37.923192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:37.923192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:37.923192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:37.923192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:37.923192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:37.923192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:37.923192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:37.923192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 279/546] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:10:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:37.927897 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b1daf-3d36-4780-b62a-2b7e63afc265 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:37.957212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29363d-a7af-44d2-bd39-7e0a44ae1741 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:38.062428 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29363d-a7af-44d2-bd39-7e0a44ae1741 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:38.068366 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:38.076204 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b1daf-3d36-4780-b62a-2b7e63afc265 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:38.099039 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e0f18-da54-4a68-8b69-de6fd975fe4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:38.100053 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e0f18-da54-4a68-8b69-de6fd975fe4b None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:38.100539 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e0f18-da54-4a68-8b69-de6fd975fe4b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:38.101023 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e0f18-da54-4a68-8b69-de6fd975fe4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:38.111484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e0f18-da54-4a68-8b69-de6fd975fe4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:38.117051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-fa0e0f18-da54-4a68-8b69-de6fd975fe4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:38.117051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:38.117051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:38.117051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:38.117051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:38.117051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:38.117051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:38.117051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:38.117051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:38.117051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:38.117051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:38.117051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:38.117051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:38.117051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:38.117051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:38.117051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:38.117936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:38.117936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:38.117936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:38.117936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:38.117936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:38.117936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:38.117936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:38.117936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:38.117936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:38.117936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:38.117936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:38.117936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:38.117936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:38.117936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:38.117936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:38.117936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:38.118809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:38.118809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:38.118809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:38.118809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:38.118809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:38.118809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:38.118809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:38.118809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:38.118809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:38.118809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:38.118809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:38.118809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:38.121497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 280/547] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:10:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:10:38.133571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e0f18-da54-4a68-8b69-de6fd975fe4b None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:38.136486 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a374f-f06b-4806-9fd0-f44b6b5d3de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:38.137095 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a374f-f06b-4806-9fd0-f44b6b5d3de2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:38.137379 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a374f-f06b-4806-9fd0-f44b6b5d3de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:38.137652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a374f-f06b-4806-9fd0-f44b6b5d3de2 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:38.138324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a374f-f06b-4806-9fd0-f44b6b5d3de2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:38.138630 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a374f-f06b-4806-9fd0-f44b6b5d3de2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:38.140149 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a374f-f06b-4806-9fd0-f44b6b5d3de2 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:38.149126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 268/548] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:10:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:38.157164 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a374f-f06b-4806-9fd0-f44b6b5d3de2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:38.164116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:38.315293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e0f18-da54-4a68-8b69-de6fd975fe4b None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:38.333275 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a374f-f06b-4806-9fd0-f44b6b5d3de2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:38.385477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48af164-a520-4738-a20c-04192a5f45cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:38.387454 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48af164-a520-4738-a20c-04192a5f45cc None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:38.387945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48af164-a520-4738-a20c-04192a5f45cc None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:38.388534 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48af164-a520-4738-a20c-04192a5f45cc None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:38.389533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48af164-a520-4738-a20c-04192a5f45cc None admin] RBAC: Adding query filter params (name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:38.390770 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48af164-a520-4738-a20c-04192a5f45cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:38.393185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48af164-a520-4738-a20c-04192a5f45cc None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:38.409416 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74c149-6edc-4c5e-a4c4-a5c09001268f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:38.410328 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74c149-6edc-4c5e-a4c4-a5c09001268f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:38.410756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74c149-6edc-4c5e-a4c4-a5c09001268f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:38.412193 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74c149-6edc-4c5e-a4c4-a5c09001268f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:38.412758 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 281/549] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:10:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:38.418561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48af164-a520-4738-a20c-04192a5f45cc None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:38.430141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74c149-6edc-4c5e-a4c4-a5c09001268f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:38.432547 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74c149-6edc-4c5e-a4c4-a5c09001268f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:38.463190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-ad74c149-6edc-4c5e-a4c4-a5c09001268f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:38.463190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:38.463190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:38.463190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:38.463190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:38.463190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:38.463190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:38.463190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:38.463190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:38.463190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:38.463190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:38.463190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:38.463190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:38.463190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:38.463190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:38.463190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:38.464396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:38.464396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:38.464396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:38.464396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:38.464396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:38.464396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:38.464396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:38.464396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:38.464396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:38.464396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:38.464396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:38.464396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:38.464396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:38.464396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:38.464396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:38.464396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:38.464396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:38.465260 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:38.465260 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:38.465260 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:38.469806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 269/550] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:10:38.475315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74c149-6edc-4c5e-a4c4-a5c09001268f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:38.490279 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:38.580553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:38.621719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74c149-6edc-4c5e-a4c4-a5c09001268f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:38.626985 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48af164-a520-4738-a20c-04192a5f45cc None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:38.696104 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac99eda-7fae-4e01-95be-1a57f81ff84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:38.697607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac99eda-7fae-4e01-95be-1a57f81ff84a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:38.698459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac99eda-7fae-4e01-95be-1a57f81ff84a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:38.699484 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac99eda-7fae-4e01-95be-1a57f81ff84a None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:38.700830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac99eda-7fae-4e01-95be-1a57f81ff84a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:38.701782 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac99eda-7fae-4e01-95be-1a57f81ff84a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:38.704511 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac99eda-7fae-4e01-95be-1a57f81ff84a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:38.705379 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efcd278-d90f-45c4-be81-7bba40691e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:38.706243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efcd278-d90f-45c4-be81-7bba40691e63 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:38.706736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efcd278-d90f-45c4-be81-7bba40691e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:38.707205 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efcd278-d90f-45c4-be81-7bba40691e63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:38.730062 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efcd278-d90f-45c4-be81-7bba40691e63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:38.732575 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 270/551] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:38.733366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efcd278-d90f-45c4-be81-7bba40691e63 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:38.739890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac99eda-7fae-4e01-95be-1a57f81ff84a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:38.764571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-9efcd278-d90f-45c4-be81-7bba40691e63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:38.764571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:38.764571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:38.764571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:38.764571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:38.764571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:38.764571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:38.764571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:38.764571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:38.764571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:38.764571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:38.764571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:38.764571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:38.764571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:38.764571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:38.764571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:38.766241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:38.766241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:38.766241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:38.766241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:38.766241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:38.766241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:38.766241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:38.766241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:38.766241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:38.766241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:38.766241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:38.766241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:38.766241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:38.766241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:38.766241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:38.766241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:38.766241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:38.767315 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:38.767315 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:38.767315 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:38.771248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 282/552] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:38.777689 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efcd278-d90f-45c4-be81-7bba40691e63 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:38.836892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:38.879048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac99eda-7fae-4e01-95be-1a57f81ff84a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:38.894521 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efcd278-d90f-45c4-be81-7bba40691e63 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:38.917013 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11975dc8-2330-4e48-b61a-ee9c0d0de668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:38.917642 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11975dc8-2330-4e48-b61a-ee9c0d0de668 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:38.917931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11975dc8-2330-4e48-b61a-ee9c0d0de668 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:38.918189 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11975dc8-2330-4e48-b61a-ee9c0d0de668 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:38.923893 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:38.926252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11975dc8-2330-4e48-b61a-ee9c0d0de668 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:38.927753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11975dc8-2330-4e48-b61a-ee9c0d0de668 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:38.933069 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa3dc0-c826-4a85-a318-7c833e2ff5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:38.933669 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa3dc0-c826-4a85-a318-7c833e2ff5d6 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:38.933946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa3dc0-c826-4a85-a318-7c833e2ff5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:38.934214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa3dc0-c826-4a85-a318-7c833e2ff5d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:38.934845 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa3dc0-c826-4a85-a318-7c833e2ff5d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:38.935150 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa3dc0-c826-4a85-a318-7c833e2ff5d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:38.936560 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa3dc0-c826-4a85-a318-7c833e2ff5d6 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:38.939430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-11975dc8-2330-4e48-b61a-ee9c0d0de668 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:10:38.939430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:38.939430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:38.939430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:38.939430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:38.939430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:38.939430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:38.939430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:38.939430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:38.939430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:38.939430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:38.939430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:38.939430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:10:38.939430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:10:38.939430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:10:38.939430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:10:38.944233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:38.944233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:38.944233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:10:38.944233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:38.944233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:10:38.944233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:38.944233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:38.944233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:38.944233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:38.944233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:38.944233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:38.944233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:38.944233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:38.944233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:38.944233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:38.944233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:38.944233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:38.944233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:38.945169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:38.945169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:38.945169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:10:38.945169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:10:38.945169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:10:38.945169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:10:38.945169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:10:38.945169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:10:38.945169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:10:38.945169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:38.945169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 271/553] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:10:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:10:38.945169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4f21d-001b-4599-928e-ba0281dea55a None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:38.945169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4f21d-001b-4599-928e-ba0281dea55a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:38.945865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4f21d-001b-4599-928e-ba0281dea55a None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:38.945865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 272/554] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:38.945865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbee70-a210-42fb-915c-e7cd1bbb709b None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:38.945865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbee70-a210-42fb-915c-e7cd1bbb709b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:38.945865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbee70-a210-42fb-915c-e7cd1bbb709b None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:38.947793 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 273/555] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:38.949419 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe1620-2934-4a62-8f43-db01ab1f8b30 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:38.949906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe1620-2934-4a62-8f43-db01ab1f8b30 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:38.950366 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe1620-2934-4a62-8f43-db01ab1f8b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:38.956628 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 283/556] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:38.959872 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa3dc0-c826-4a85-a318-7c833e2ff5d6 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:38.970012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-98fe1620-2934-4a62-8f43-db01ab1f8b30 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:38.971564 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-98fe1620-2934-4a62-8f43-db01ab1f8b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:39.051792 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 274/557] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:10:39.051792 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc6715-76af-4ff0-8c8f-df8c9cf0f196 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:39.051792 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc6715-76af-4ff0-8c8f-df8c9cf0f196 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:39.051792 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc6715-76af-4ff0-8c8f-df8c9cf0f196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:39.061011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-77cc6715-76af-4ff0-8c8f-df8c9cf0f196 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:39.062295 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-77cc6715-76af-4ff0-8c8f-df8c9cf0f196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:39.083124 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa3dc0-c826-4a85-a318-7c833e2ff5d6 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:39.120459 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088c622-87c4-4117-a4fd-c5ca25784431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:39.121057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088c622-87c4-4117-a4fd-c5ca25784431 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:39.121348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088c622-87c4-4117-a4fd-c5ca25784431 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:39.121589 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088c622-87c4-4117-a4fd-c5ca25784431 None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:39.122359 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088c622-87c4-4117-a4fd-c5ca25784431 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:39.122660 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088c622-87c4-4117-a4fd-c5ca25784431 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:39.124198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088c622-87c4-4117-a4fd-c5ca25784431 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:39.132713 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:39.148170 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 284/558] 10.222.0.81 () {56 vars in 1135 bytes} [Mon Sep 19 11:10:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:39.149134 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088c622-87c4-4117-a4fd-c5ca25784431 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:39.151448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 275/559] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:39.152462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc6715-76af-4ff0-8c8f-df8c9cf0f196 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:39.206513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:39.266214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088c622-87c4-4117-a4fd-c5ca25784431 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:39.287818 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc6715-76af-4ff0-8c8f-df8c9cf0f196 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:39.305419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980acd2-5118-43eb-bd57-0c253ff6e0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:39.306067 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980acd2-5118-43eb-bd57-0c253ff6e0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:39.306341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980acd2-5118-43eb-bd57-0c253ff6e0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:39.306608 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980acd2-5118-43eb-bd57-0c253ff6e0b0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:39.314078 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980acd2-5118-43eb-bd57-0c253ff6e0b0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:39.315553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980acd2-5118-43eb-bd57-0c253ff6e0b0 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:39.327470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-9980acd2-5118-43eb-bd57-0c253ff6e0b0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:10:39.327470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:39.327470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:39.327470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:39.327470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:39.327470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:39.327470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:39.327470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:39.327470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:39.327470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:39.327470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:39.327470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:39.327470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:10:39.327470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:10:39.327470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:10:39.327470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:10:39.328740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:39.328740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:39.328740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:10:39.328740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:39.328740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:10:39.328740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:39.328740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:39.328740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:39.328740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:39.328740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:39.328740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:39.328740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:39.328740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:39.328740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:39.328740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:39.328740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:39.328740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:39.328740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:39.329707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:39.329707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:39.329707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:10:39.329707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:10:39.329707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:10:39.329707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:10:39.329707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:10:39.329707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:10:39.329707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:10:39.329707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:39.329707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 285/560] 10.222.0.81 () {56 vars in 1093 bytes} [Mon Sep 19 11:10:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:39.335242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980acd2-5118-43eb-bd57-0c253ff6e0b0 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:39.393195 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2653b81-68d5-44c2-9727-2100badf9b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:39.394203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2653b81-68d5-44c2-9727-2100badf9b28 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:39.394784 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2653b81-68d5-44c2-9727-2100badf9b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:39.395263 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2653b81-68d5-44c2-9727-2100badf9b28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:39.448157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980acd2-5118-43eb-bd57-0c253ff6e0b0 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:39.454347 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:39.457415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:39.467719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2653b81-68d5-44c2-9727-2100badf9b28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:39.469219 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2653b81-68d5-44c2-9727-2100badf9b28 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:39.486719 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bac254-65e1-4816-b00f-cbd5c20a3a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:39.487312 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bac254-65e1-4816-b00f-cbd5c20a3a31 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:39.487588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bac254-65e1-4816-b00f-cbd5c20a3a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:39.487884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bac254-65e1-4816-b00f-cbd5c20a3a31 None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:39.488602 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bac254-65e1-4816-b00f-cbd5c20a3a31 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:39.488905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bac254-65e1-4816-b00f-cbd5c20a3a31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:39.490312 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bac254-65e1-4816-b00f-cbd5c20a3a31 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:39.508291 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 286/561] 10.222.0.81 () {56 vars in 1133 bytes} [Mon Sep 19 11:10:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:39.509642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bac254-65e1-4816-b00f-cbd5c20a3a31 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:39.571325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 276/562] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:10:39.573213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d454a-6da8-4bd9-9089-38d71a26256f None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:39.573501 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d454a-6da8-4bd9-9089-38d71a26256f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:39.573773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d454a-6da8-4bd9-9089-38d71a26256f None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:39.579520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 277/563] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:39.580371 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d454a-6da8-4bd9-9089-38d71a26256f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:39.609239 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bac254-65e1-4816-b00f-cbd5c20a3a31 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:39.643236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08297688-7643-444b-b67b-24458e615e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:39.643905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08297688-7643-444b-b67b-24458e615e19 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:39.644185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08297688-7643-444b-b67b-24458e615e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:39.644459 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08297688-7643-444b-b67b-24458e615e19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:39.656682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08297688-7643-444b-b67b-24458e615e19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:39.661478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-08297688-7643-444b-b67b-24458e615e19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:39.661478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:39.661478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:39.661478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:39.661478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:39.661478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:39.661478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:39.661478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:39.661478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:39.661478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:39.661478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:39.661478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:39.661478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:39.661478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:39.661478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:39.661478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:39.662569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:39.662569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:39.662569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:39.662569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:39.662569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:39.662569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:39.662569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:39.662569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:39.662569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:39.662569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:39.662569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:39.662569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:39.662569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:39.662569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:39.662569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:39.662569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:39.663505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:39.663505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:39.663505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:39.663505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:39.663505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:39.663505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:39.663505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:39.663505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:39.663505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:39.663505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:39.663505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:39.663505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:39.663505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 287/564] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:10:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:39.663505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08297688-7643-444b-b67b-24458e615e19 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:39.673178 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:39.688397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d454a-6da8-4bd9-9089-38d71a26256f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:39.690997 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:39.723532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cbded-1103-47e3-b6c6-2ea27ea1b74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:39.724114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cbded-1103-47e3-b6c6-2ea27ea1b74f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:39.724392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cbded-1103-47e3-b6c6-2ea27ea1b74f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:39.724663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cbded-1103-47e3-b6c6-2ea27ea1b74f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:39.734912 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77cbded-1103-47e3-b6c6-2ea27ea1b74f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:39.736401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77cbded-1103-47e3-b6c6-2ea27ea1b74f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:39.750699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-f77cbded-1103-47e3-b6c6-2ea27ea1b74f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:39.750699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:39.750699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:39.750699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:39.750699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:39.750699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:39.750699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:39.750699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:39.750699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:39.750699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:39.750699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:39.750699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:39.750699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:39.750699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:39.750699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:39.750699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:39.751698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:39.751698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:39.751698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:39.751698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:39.751698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:39.751698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:39.751698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:39.751698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:39.751698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:39.751698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:39.751698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:39.751698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:39.751698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:39.751698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:39.751698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:39.751698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:39.751698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:39.752633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:39.752633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:39.752633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:39.752633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 278/565] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:39.752633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cbded-1103-47e3-b6c6-2ea27ea1b74f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:39.769281 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08297688-7643-444b-b67b-24458e615e19 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:39.803324 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798d5e5-a3da-46e4-b024-95ee17d1c94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:39.803965 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798d5e5-a3da-46e4-b024-95ee17d1c94c None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:39.804241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798d5e5-a3da-46e4-b024-95ee17d1c94c None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:39.804506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798d5e5-a3da-46e4-b024-95ee17d1c94c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:39.814718 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798d5e5-a3da-46e4-b024-95ee17d1c94c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:39.819398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-1798d5e5-a3da-46e4-b024-95ee17d1c94c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:39.819398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:39.819398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:39.819398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:39.819398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:39.819398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:39.819398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:39.819398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:39.819398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:39.819398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:39.819398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:39.819398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:39.819398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:39.819398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:39.819398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:39.819398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:39.820361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:39.820361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:39.820361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:39.820361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:39.820361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:39.820361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:39.820361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:39.820361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:39.820361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:39.820361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:39.820361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:39.820361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:39.820361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:39.820361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:39.820361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:39.820361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:39.821204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:39.821204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:39.821204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:39.821204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:39.821204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:39.821204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:39.821204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:39.821204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:39.821204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:39.821204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:39.821204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:39.821204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:39.821204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 288/566] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:10:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:39.822474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb13646-ea21-4a04-a375-e7bacc6a3c69 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:39.822474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb13646-ea21-4a04-a375-e7bacc6a3c69 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:39.822711 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb13646-ea21-4a04-a375-e7bacc6a3c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:39.839956 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-cbb13646-ea21-4a04-a375-e7bacc6a3c69 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:39.841143 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-cbb13646-ea21-4a04-a375-e7bacc6a3c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:39.854215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cbded-1103-47e3-b6c6-2ea27ea1b74f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:39.887422 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b85895-875d-4d42-b574-0130b7317502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:39.888079 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b85895-875d-4d42-b574-0130b7317502 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:39.888365 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b85895-875d-4d42-b574-0130b7317502 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:39.888640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b85895-875d-4d42-b574-0130b7317502 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:39.897368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:39.899848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b85895-875d-4d42-b574-0130b7317502 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:39.901609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b85895-875d-4d42-b574-0130b7317502 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:39.906277 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 289/567] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:39.907111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb13646-ea21-4a04-a375-e7bacc6a3c69 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:39.922285 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:39.928235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-33b85895-875d-4d42-b574-0130b7317502 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:39.928235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:39.928235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:39.928235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:39.928235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:39.928235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:39.928235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:39.928235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:39.928235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:39.928235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:39.928235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:39.928235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:39.928235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:39.928235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:39.928235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:39.928235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:39.929160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:39.929160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:39.929160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:39.929160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:39.929160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:39.929160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:39.929160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:39.929160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:39.929160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:39.929160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:39.929160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:39.929160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:39.929160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:39.929160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:39.929160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:39.929160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:39.929160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:39.930038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:39.930038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:39.930038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:39.930038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 279/568] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:39.930709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b85895-875d-4d42-b574-0130b7317502 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:40.014440 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb13646-ea21-4a04-a375-e7bacc6a3c69 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:40.045558 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b85895-875d-4d42-b574-0130b7317502 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:40.049066 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae93c50-bfd9-4dec-95c4-a8aafa1070b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:40.049665 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae93c50-bfd9-4dec-95c4-a8aafa1070b5 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:40.049942 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae93c50-bfd9-4dec-95c4-a8aafa1070b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:40.050213 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae93c50-bfd9-4dec-95c4-a8aafa1070b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:40.050894 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae93c50-bfd9-4dec-95c4-a8aafa1070b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:40.051196 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae93c50-bfd9-4dec-95c4-a8aafa1070b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:40.052591 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae93c50-bfd9-4dec-95c4-a8aafa1070b5 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:40.061176 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 290/569] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:10:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:40.066199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae93c50-bfd9-4dec-95c4-a8aafa1070b5 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:40.086556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e35974-63a2-4b54-a781-cbe8c68d2c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:40.087933 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e35974-63a2-4b54-a781-cbe8c68d2c6c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:40.088825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e35974-63a2-4b54-a781-cbe8c68d2c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:40.089970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e35974-63a2-4b54-a781-cbe8c68d2c6c None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:40.092130 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e35974-63a2-4b54-a781-cbe8c68d2c6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:40.092852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e35974-63a2-4b54-a781-cbe8c68d2c6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:40.093701 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e35974-63a2-4b54-a781-cbe8c68d2c6c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:40.105846 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 280/570] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:40.106577 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e35974-63a2-4b54-a781-cbe8c68d2c6c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:40.158368 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:40.162937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:40.187463 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae93c50-bfd9-4dec-95c4-a8aafa1070b5 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:40.209990 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e35974-63a2-4b54-a781-cbe8c68d2c6c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:40.220611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29215ebb-842d-44d8-b8c7-2bef333156eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:40.221205 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29215ebb-842d-44d8-b8c7-2bef333156eb None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:40.221483 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29215ebb-842d-44d8-b8c7-2bef333156eb None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:40.221758 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29215ebb-842d-44d8-b8c7-2bef333156eb None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:40.222417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29215ebb-842d-44d8-b8c7-2bef333156eb None admin] RBAC: Adding query filter params (name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:40.222720 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29215ebb-842d-44d8-b8c7-2bef333156eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:40.224238 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29215ebb-842d-44d8-b8c7-2bef333156eb None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:40.233029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 291/571] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:10:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:40.233813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29215ebb-842d-44d8-b8c7-2bef333156eb None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:40.246635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da87295-3d40-4ad6-b6a7-dafd5ead29f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:40.247241 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da87295-3d40-4ad6-b6a7-dafd5ead29f5 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:40.247524 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da87295-3d40-4ad6-b6a7-dafd5ead29f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:40.247823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da87295-3d40-4ad6-b6a7-dafd5ead29f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:40.248476 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da87295-3d40-4ad6-b6a7-dafd5ead29f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:40.248767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da87295-3d40-4ad6-b6a7-dafd5ead29f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:40.250143 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da87295-3d40-4ad6-b6a7-dafd5ead29f5 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:40.261637 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 281/572] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:40.262419 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da87295-3d40-4ad6-b6a7-dafd5ead29f5 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:40.346479 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29215ebb-842d-44d8-b8c7-2bef333156eb None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:40.363644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da87295-3d40-4ad6-b6a7-dafd5ead29f5 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:40.376188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:40.379737 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f47151-f31e-4b99-855f-25170ec8598d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:40.380382 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f47151-f31e-4b99-855f-25170ec8598d None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:40.380657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f47151-f31e-4b99-855f-25170ec8598d None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:40.380931 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f47151-f31e-4b99-855f-25170ec8598d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:40.388268 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:40.392464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f47151-f31e-4b99-855f-25170ec8598d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:40.393898 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f47151-f31e-4b99-855f-25170ec8598d None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:40.397917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0990c-b916-47c3-8733-ec812382672a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:40.398533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0990c-b916-47c3-8733-ec812382672a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:40.398812 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0990c-b916-47c3-8733-ec812382672a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:40.399084 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0990c-b916-47c3-8733-ec812382672a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:40.406454 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0990c-b916-47c3-8733-ec812382672a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:40.407930 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0990c-b916-47c3-8733-ec812382672a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:40.408421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-19f47151-f31e-4b99-855f-25170ec8598d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:40.408421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:40.408421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:40.408421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:40.408421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:40.408421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:40.408421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:40.408421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:40.408421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:40.408421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:40.408421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:40.408421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:40.408421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:40.408421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:40.408421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:40.408421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:40.409318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:40.409318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:40.409318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:40.409318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:40.409318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:40.409318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:40.409318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:40.409318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:40.409318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:40.409318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:40.409318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:40.409318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:40.409318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:40.409318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:40.409318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:40.409318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:40.409318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:40.410214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:40.410214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:40.410214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:40.410214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 292/573] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:40.410585 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f47151-f31e-4b99-855f-25170ec8598d None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:40.422754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-37f0990c-b916-47c3-8733-ec812382672a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:40.422754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:40.422754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:40.422754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:40.422754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:40.422754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:40.422754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:40.422754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:40.422754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:40.422754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:40.422754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:40.422754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:40.422754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:10:40.422754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:10:40.422754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:10:40.423702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:10:40.423702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:40.423702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:40.423702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:40.423702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:40.423702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:40.423702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:40.423702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:40.423702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:40.423702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:40.423702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:40.423702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:40.423702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:40.423702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:40.423702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:40.423702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:40.423702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:40.424642 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:10:40.424642 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:10:40.424642 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:10:40.424642 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:10:40.424642 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:10:40.424642 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:10:40.424642 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:40.424642 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:40.424642 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 282/574] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:40.424642 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0990c-b916-47c3-8733-ec812382672a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:40.510981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f47151-f31e-4b99-855f-25170ec8598d None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:40.525426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0990c-b916-47c3-8733-ec812382672a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:40.544105 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787bd4d-dbab-4dcb-bff1-52b1a865f565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:40.544729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787bd4d-dbab-4dcb-bff1-52b1a865f565 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:40.545010 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787bd4d-dbab-4dcb-bff1-52b1a865f565 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:40.545286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787bd4d-dbab-4dcb-bff1-52b1a865f565 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:40.555138 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0787bd4d-dbab-4dcb-bff1-52b1a865f565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:40.558989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3484c4-8826-41ca-a0c7-d04cb9a74f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:40.559594 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3484c4-8826-41ca-a0c7-d04cb9a74f9a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:40.559973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3484c4-8826-41ca-a0c7-d04cb9a74f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:40.560323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3484c4-8826-41ca-a0c7-d04cb9a74f9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:40.561055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-0787bd4d-dbab-4dcb-bff1-52b1a865f565 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:40.561055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:40.561055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:40.561055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:40.561055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:40.561055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:40.561055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:40.561055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:40.561055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:40.561055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:40.561055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:40.561055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:40.561055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:40.561055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:40.561055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:40.561055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:40.562029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:40.562029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:40.562029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:40.562029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:40.562029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:40.562029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:40.562029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:40.562029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:40.562029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:40.562029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:40.562029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:40.562029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:40.562029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:40.562029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:40.562029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:40.562029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:40.562873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:40.562873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:40.562873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:40.562873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:40.562873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:40.562873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:40.562873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:40.562873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:40.562873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:40.562873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:40.562873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:40.562873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:40.562873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 293/575] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:10:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:40.564148 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787bd4d-dbab-4dcb-bff1-52b1a865f565 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:40.572116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3484c4-8826-41ca-a0c7-d04cb9a74f9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:40.573713 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3484c4-8826-41ca-a0c7-d04cb9a74f9a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:40.588161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-7b3484c4-8826-41ca-a0c7-d04cb9a74f9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:40.588161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:40.588161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:40.588161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:40.588161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:40.588161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:40.588161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:40.588161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:40.588161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:40.588161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:40.588161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:40.588161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:40.588161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:40.588161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:40.588161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:40.588161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:40.589287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:40.589287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:40.589287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:40.589287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:40.589287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:40.589287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:40.589287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:40.589287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:40.589287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:40.589287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:40.589287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:40.589287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:40.589287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:40.589287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:40.589287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:40.589287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:40.589287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:40.590443 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:40.590443 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:40.590443 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:40.593776 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 283/576] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:40.594552 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3484c4-8826-41ca-a0c7-d04cb9a74f9a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:40.618195 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:40.619570 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:40.670523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787bd4d-dbab-4dcb-bff1-52b1a865f565 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:40.693950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3484c4-8826-41ca-a0c7-d04cb9a74f9a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:40.705749 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3989650-0a6e-4c08-8617-b1e6e77b5c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:40.706377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3989650-0a6e-4c08-8617-b1e6e77b5c91 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:40.706665 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3989650-0a6e-4c08-8617-b1e6e77b5c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:40.706940 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3989650-0a6e-4c08-8617-b1e6e77b5c91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:40.714125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3989650-0a6e-4c08-8617-b1e6e77b5c91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:40.715551 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3989650-0a6e-4c08-8617-b1e6e77b5c91 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:40.726575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-a3989650-0a6e-4c08-8617-b1e6e77b5c91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:40.726575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:40.726575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:40.726575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:40.726575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:40.726575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:40.726575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:40.726575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:40.726575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:40.726575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:40.726575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:40.726575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:40.726575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:10:40.726575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:10:40.726575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:10:40.727649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:10:40.727649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:40.727649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:40.727649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:40.727649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:40.727649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:40.727649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:40.727649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:40.727649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:40.727649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:40.727649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:40.727649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:40.727649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:40.727649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:40.727649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:40.727649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:40.727649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:40.729141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:10:40.729141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:10:40.729141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:10:40.729141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:10:40.729141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:10:40.729141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:10:40.729141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:40.729141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:40.729686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1c937-43ea-456d-9761-f6ce335ef52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:40.729785 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 294/577] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:40.729858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1c937-43ea-456d-9761-f6ce335ef52f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:40.729858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1c937-43ea-456d-9761-f6ce335ef52f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:40.729858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1c937-43ea-456d-9761-f6ce335ef52f None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:40.730044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3989650-0a6e-4c08-8617-b1e6e77b5c91 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:40.730114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1c937-43ea-456d-9761-f6ce335ef52f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:40.730114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1c937-43ea-456d-9761-f6ce335ef52f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:40.731157 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1c937-43ea-456d-9761-f6ce335ef52f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:40.742700 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 284/578] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:40.744057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1c937-43ea-456d-9761-f6ce335ef52f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:40.824014 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:40.837969 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3989650-0a6e-4c08-8617-b1e6e77b5c91 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:40.840835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1c937-43ea-456d-9761-f6ce335ef52f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:40.847970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:40.871525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc40d8c-be58-42e7-b8b8-b3f7b5a6f7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:40.872082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc40d8c-be58-42e7-b8b8-b3f7b5a6f7ca None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:40.872360 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc40d8c-be58-42e7-b8b8-b3f7b5a6f7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:40.872632 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc40d8c-be58-42e7-b8b8-b3f7b5a6f7ca None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:40.873313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc40d8c-be58-42e7-b8b8-b3f7b5a6f7ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:40.873616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc40d8c-be58-42e7-b8b8-b3f7b5a6f7ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:40.874610 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c02e6-670d-4d2e-869d-03513fe3e148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:40.875019 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc40d8c-be58-42e7-b8b8-b3f7b5a6f7ca None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:40.875206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c02e6-670d-4d2e-869d-03513fe3e148 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:40.875489 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c02e6-670d-4d2e-869d-03513fe3e148 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:40.875798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c02e6-670d-4d2e-869d-03513fe3e148 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:40.876449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c02e6-670d-4d2e-869d-03513fe3e148 None admin] RBAC: Adding query filter params (name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:40.876757 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c02e6-670d-4d2e-869d-03513fe3e148 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:40.878112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c02e6-670d-4d2e-869d-03513fe3e148 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:40.885882 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 285/579] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:10:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:40.886659 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c02e6-670d-4d2e-869d-03513fe3e148 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:40.891334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 295/580] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:10:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:40.892259 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc40d8c-be58-42e7-b8b8-b3f7b5a6f7ca None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:40.984854 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c02e6-670d-4d2e-869d-03513fe3e148 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:40.991720 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc40d8c-be58-42e7-b8b8-b3f7b5a6f7ca None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:41.017387 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372689b-3371-4f79-b243-5ba15b9633f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:41.017990 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372689b-3371-4f79-b243-5ba15b9633f2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:41.018258 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372689b-3371-4f79-b243-5ba15b9633f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:41.018529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372689b-3371-4f79-b243-5ba15b9633f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:41.019162 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372689b-3371-4f79-b243-5ba15b9633f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:41.019465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372689b-3371-4f79-b243-5ba15b9633f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:41.020867 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372689b-3371-4f79-b243-5ba15b9633f2 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:41.025438 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d91930-035a-447a-8c36-1297598b9766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:41.026027 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d91930-035a-447a-8c36-1297598b9766 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:41.026302 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d91930-035a-447a-8c36-1297598b9766 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:41.026568 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d91930-035a-447a-8c36-1297598b9766 None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:41.027247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d91930-035a-447a-8c36-1297598b9766 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:41.027548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d91930-035a-447a-8c36-1297598b9766 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:41.028980 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d91930-035a-447a-8c36-1297598b9766 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:41.037104 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 286/581] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:41.037104 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372689b-3371-4f79-b243-5ba15b9633f2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:41.040676 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 296/582] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:10:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:41.041642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d91930-035a-447a-8c36-1297598b9766 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:41.046795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:41.070678 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:41.139129 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372689b-3371-4f79-b243-5ba15b9633f2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:41.139297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d91930-035a-447a-8c36-1297598b9766 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:41.171927 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583f8f8-e2e9-44a6-a7b5-3de3aaa514c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:41.172314 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe7a71-7a67-4487-bee6-554c3d0bb333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:41.172544 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583f8f8-e2e9-44a6-a7b5-3de3aaa514c9 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:41.172822 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583f8f8-e2e9-44a6-a7b5-3de3aaa514c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:41.172929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe7a71-7a67-4487-bee6-554c3d0bb333 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:41.173094 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583f8f8-e2e9-44a6-a7b5-3de3aaa514c9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:41.173197 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe7a71-7a67-4487-bee6-554c3d0bb333 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:41.173452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe7a71-7a67-4487-bee6-554c3d0bb333 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:41.179877 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583f8f8-e2e9-44a6-a7b5-3de3aaa514c9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:41.181868 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583f8f8-e2e9-44a6-a7b5-3de3aaa514c9 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:41.184384 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe7a71-7a67-4487-bee6-554c3d0bb333 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:41.185850 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe7a71-7a67-4487-bee6-554c3d0bb333 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:41.193229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-7583f8f8-e2e9-44a6-a7b5-3de3aaa514c9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:10:41.193229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:41.193229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:41.193229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:41.193229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:41.193229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:41.193229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:41.193229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:41.193229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:41.193229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:41.193229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:41.193229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:41.193229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:10:41.193229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:10:41.193229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:10:41.193229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:10:41.194173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:41.194173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:41.194173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:10:41.194173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:41.194173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:10:41.194173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:41.194173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:41.194173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:41.194173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:41.194173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:41.194173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:41.194173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:41.194173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:41.194173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:41.194173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:41.194173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:41.194173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:41.194173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:41.195146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:41.195146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:41.195146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:10:41.195146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:10:41.195146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:10:41.195146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:10:41.195146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:10:41.195146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:10:41.195146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:10:41.195146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:41.195146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 287/583] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:41.195146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583f8f8-e2e9-44a6-a7b5-3de3aaa514c9 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:41.202397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-2afe7a71-7a67-4487-bee6-554c3d0bb333 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:41.202397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:41.202397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:41.202397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:41.202397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:41.202397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:41.202397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:41.202397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:41.202397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:41.202397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:41.202397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:41.202397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:41.202397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:41.202397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:41.202397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:41.202397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:41.203783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:41.203783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:41.203783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:41.203783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:41.203783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:41.203783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:41.203783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:41.203783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:41.203783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:41.203783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:41.203783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:41.203783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:41.203783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:41.203783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:41.203783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:41.203783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:41.203783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:41.204690 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:41.204690 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:41.204690 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:41.204690 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 297/584] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:41.204690 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe7a71-7a67-4487-bee6-554c3d0bb333 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:41.260671 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:41.286457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:41.296455 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583f8f8-e2e9-44a6-a7b5-3de3aaa514c9 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:41.305371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe7a71-7a67-4487-bee6-554c3d0bb333 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:41.329169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9537d-8553-4efb-be93-9818a3fb2b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:41.329819 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9537d-8553-4efb-be93-9818a3fb2b43 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:41.330100 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9537d-8553-4efb-be93-9818a3fb2b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:41.330375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9537d-8553-4efb-be93-9818a3fb2b43 None admin] PATH_INFO: `/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/5abd70a18ccd407bafe3b61c8870986e/roles/af9d39307d714527b14b2045ab466e7a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:41.337917 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c89a2d-40db-4d8c-923c-455fc835f432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:41.338528 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c89a2d-40db-4d8c-923c-455fc835f432 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:41.338807 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c89a2d-40db-4d8c-923c-455fc835f432 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:41.339077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c89a2d-40db-4d8c-923c-455fc835f432 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:41.344802 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9537d-8553-4efb-be93-9818a3fb2b43 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed9a0b626e3e4b93b391158e981ec586, user_id=5abd70a18ccd407bafe3b61c8870986e, role_id=af9d39307d714527b14b2045ab466e7a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:41.346243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c89a2d-40db-4d8c-923c-455fc835f432 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:41.346371 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9537d-8553-4efb-be93-9818a3fb2b43 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:41.347649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c89a2d-40db-4d8c-923c-455fc835f432 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:41.359005 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 288/585] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:10:41 2022] PUT /identity/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/5abd70a18ccd407bafe3b61c8870986e/roles/af9d39307d714527b14b2045ab466e7a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:10:41.359371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-a4c89a2d-40db-4d8c-923c-455fc835f432 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 19 11:10:41.359371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:41.359371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:41.359371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:41.359371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:41.359371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:41.359371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:41.359371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:41.359371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:41.359371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:41.359371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:41.359371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:41.359371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:10:41.359371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:10:41.359371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:10:41.359371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:10:41.360520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:41.360520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:41.360520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:10:41.360520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:41.360520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:10:41.360520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:41.360520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:41.360520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:41.360520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:41.360520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:41.360520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:41.360520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:41.360520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:41.360520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:41.360520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:41.360520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:41.360520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:41.360520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:41.361470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:41.361470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:41.361470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:10:41.361470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:10:41.361470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:10:41.361470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:10:41.361470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:10:41.361470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:10:41.361470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 19 11:10:41.361470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:41.361470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 298/586] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:41.362065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c89a2d-40db-4d8c-923c-455fc835f432 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:41.362887 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9537d-8553-4efb-be93-9818a3fb2b43 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:41.478914 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c89a2d-40db-4d8c-923c-455fc835f432 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:41.485476 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9537d-8553-4efb-be93-9818a3fb2b43 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:41.505105 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:41.517038 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0ca19-10d1-462e-9425-7322af5f43d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:41.517648 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0ca19-10d1-462e-9425-7322af5f43d8 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:41.517927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0ca19-10d1-462e-9425-7322af5f43d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:41.518198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0ca19-10d1-462e-9425-7322af5f43d8 None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:41.519114 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0ca19-10d1-462e-9425-7322af5f43d8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:41.519792 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0ca19-10d1-462e-9425-7322af5f43d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:41.521423 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0ca19-10d1-462e-9425-7322af5f43d8 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:41.532123 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460baff-ddf3-4a44-8485-7e82ff5d62f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:41.533027 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460baff-ddf3-4a44-8485-7e82ff5d62f3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:41.533027 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460baff-ddf3-4a44-8485-7e82ff5d62f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:41.533306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460baff-ddf3-4a44-8485-7e82ff5d62f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:41.534368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460baff-ddf3-4a44-8485-7e82ff5d62f3 None admin] RBAC: Adding query filter params (role.id=af9d39307d714527b14b2045ab466e7a, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=1c8a9732807d4921a41a6a2d5019e2f5) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:41.534368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460baff-ddf3-4a44-8485-7e82ff5d62f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:41.536556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460baff-ddf3-4a44-8485-7e82ff5d62f3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:41.538755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:41.542981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 299/587] 10.222.0.81 () {56 vars in 1137 bytes} [Mon Sep 19 11:10:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:41.543962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0ca19-10d1-462e-9425-7322af5f43d8 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:41.548947 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 289/588] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:10:41 2022] GET /identity/v3/role_assignments?user.id=1c8a9732807d4921a41a6a2d5019e2f5&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=af9d39307d714527b14b2045ab466e7a => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:41.549859 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460baff-ddf3-4a44-8485-7e82ff5d62f3 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:41.688481 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0ca19-10d1-462e-9425-7322af5f43d8 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:41.716637 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460baff-ddf3-4a44-8485-7e82ff5d62f3 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:41.737900 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519647db-fc23-4464-9c61-5760ffa93826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:41.738817 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519647db-fc23-4464-9c61-5760ffa93826 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:41.738817 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519647db-fc23-4464-9c61-5760ffa93826 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:41.739095 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519647db-fc23-4464-9c61-5760ffa93826 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:41.739714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519647db-fc23-4464-9c61-5760ffa93826 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:41.740667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519647db-fc23-4464-9c61-5760ffa93826 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:41.742086 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519647db-fc23-4464-9c61-5760ffa93826 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:41.758802 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 300/589] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:41.760579 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519647db-fc23-4464-9c61-5760ffa93826 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:41.772718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f17c1-8b36-4aa0-8fa0-23449c49ef5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:41.773327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f17c1-8b36-4aa0-8fa0-23449c49ef5a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:41.773667 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f17c1-8b36-4aa0-8fa0-23449c49ef5a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:41.773944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f17c1-8b36-4aa0-8fa0-23449c49ef5a None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:41.775209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f17c1-8b36-4aa0-8fa0-23449c49ef5a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:41.775545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f17c1-8b36-4aa0-8fa0-23449c49ef5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:41.777300 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f17c1-8b36-4aa0-8fa0-23449c49ef5a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:41.801215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 290/590] 10.222.0.81 () {56 vars in 1137 bytes} [Mon Sep 19 11:10:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:41.802394 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f17c1-8b36-4aa0-8fa0-23449c49ef5a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:41.831279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:41.844247 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:41.879019 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519647db-fc23-4464-9c61-5760ffa93826 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:41.906064 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f17c1-8b36-4aa0-8fa0-23449c49ef5a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:41.923001 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac21c96-ddd8-4eee-9766-34f5d9ad6fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:41.923001 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac21c96-ddd8-4eee-9766-34f5d9ad6fcd None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:41.923001 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac21c96-ddd8-4eee-9766-34f5d9ad6fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:41.923001 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac21c96-ddd8-4eee-9766-34f5d9ad6fcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:41.931996 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac21c96-ddd8-4eee-9766-34f5d9ad6fcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:41.933971 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac21c96-ddd8-4eee-9766-34f5d9ad6fcd None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:41.950465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-8ac21c96-ddd8-4eee-9766-34f5d9ad6fcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:41.950465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:41.950465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:41.950465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:41.950465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:41.950465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:41.950465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:41.950465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:41.950465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:41.950465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:41.950465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:41.950465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:41.950465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:41.950465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:41.950465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:41.950465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:41.951410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:41.951410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:41.951410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:41.951410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:41.951410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:41.951410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:41.951410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:41.951410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:41.951410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:41.951410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:41.951410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:41.951410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:41.951410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:41.951410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:41.951410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:41.951410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:41.951410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:41.952332 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:41.952332 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:41.952332 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:41.952985 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e55dc-3bb7-42f1-bbff-d7d0aed8c40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:41.953877 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e55dc-3bb7-42f1-bbff-d7d0aed8c40e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:41.954158 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e55dc-3bb7-42f1-bbff-d7d0aed8c40e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:41.954431 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e55dc-3bb7-42f1-bbff-d7d0aed8c40e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:41.956102 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 301/591] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:41.957128 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac21c96-ddd8-4eee-9766-34f5d9ad6fcd None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:41.963710 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e55dc-3bb7-42f1-bbff-d7d0aed8c40e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:41.965263 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e55dc-3bb7-42f1-bbff-d7d0aed8c40e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:41.980148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-c28e55dc-3bb7-42f1-bbff-d7d0aed8c40e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 11:10:41.980148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:41.980148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:41.980148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:41.980148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:41.980148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:41.980148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:41.980148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:41.980148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:41.980148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:41.980148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:41.980148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:41.980148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:10:41.980148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:10:41.980148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:10:41.980148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:10:41.981089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:41.981089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:41.981089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:10:41.981089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:41.981089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:10:41.981089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:41.981089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:41.981089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:41.981089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:41.981089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:41.981089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:41.981089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:41.981089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:41.981089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:41.981089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:41.981089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:41.981089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:41.981089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:41.982019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:41.982019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:41.982019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:10:41.982019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:10:41.982019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:10:41.982019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:10:41.982019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:10:41.982019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:10:41.982019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 11:10:41.982019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:41.982019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 291/592] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:10:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:41.982598 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e55dc-3bb7-42f1-bbff-d7d0aed8c40e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:42.082114 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac21c96-ddd8-4eee-9766-34f5d9ad6fcd None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:42.096010 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:42.097655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:42.105079 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e55dc-3bb7-42f1-bbff-d7d0aed8c40e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:42.122894 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345f3f8-aa7c-4c6b-807a-415bc2d6bb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:42.123519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345f3f8-aa7c-4c6b-807a-415bc2d6bb97 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:42.123830 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345f3f8-aa7c-4c6b-807a-415bc2d6bb97 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:42.124124 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345f3f8-aa7c-4c6b-807a-415bc2d6bb97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:42.136430 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345f3f8-aa7c-4c6b-807a-415bc2d6bb97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:42.137876 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345f3f8-aa7c-4c6b-807a-415bc2d6bb97 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:42.152512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-4345f3f8-aa7c-4c6b-807a-415bc2d6bb97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:42.152512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:42.152512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:42.152512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:42.152512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:42.152512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:42.152512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:42.152512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:42.152512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:42.152512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:42.152512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:42.152512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:42.152512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:42.152512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:42.152512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:42.152512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:42.154655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:42.154655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:42.154655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:42.154655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:42.154655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:42.154655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:42.154655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:42.154655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:42.154655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:42.154655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:42.154655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:42.154655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:42.154655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:42.154655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:42.154655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:42.154655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:42.154655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:42.156780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:42.156780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:42.156780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:42.156780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 302/593] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:42.156780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345f3f8-aa7c-4c6b-807a-415bc2d6bb97 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:42.157302 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83efac4-50df-47fd-9104-763e18503537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:42.157302 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83efac4-50df-47fd-9104-763e18503537 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:42.157302 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83efac4-50df-47fd-9104-763e18503537 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:42.157302 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83efac4-50df-47fd-9104-763e18503537 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:42.157620 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83efac4-50df-47fd-9104-763e18503537 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:42.157620 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83efac4-50df-47fd-9104-763e18503537 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:42.161532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83efac4-50df-47fd-9104-763e18503537 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:42.181234 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 292/594] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:42.182145 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83efac4-50df-47fd-9104-763e18503537 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:42.299558 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83efac4-50df-47fd-9104-763e18503537 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:42.300919 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345f3f8-aa7c-4c6b-807a-415bc2d6bb97 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:42.336470 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b2330-6cee-46de-a1f6-6489b6b1835c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:42.337688 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b2330-6cee-46de-a1f6-6489b6b1835c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:42.338036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b2330-6cee-46de-a1f6-6489b6b1835c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:42.338380 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b2330-6cee-46de-a1f6-6489b6b1835c None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:42.339094 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b2330-6cee-46de-a1f6-6489b6b1835c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:42.339505 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b2330-6cee-46de-a1f6-6489b6b1835c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:42.341003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b2330-6cee-46de-a1f6-6489b6b1835c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:42.346644 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f73bd-b262-4ce3-88e2-bade18e055e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:42.347248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f73bd-b262-4ce3-88e2-bade18e055e2 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:42.347528 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f73bd-b262-4ce3-88e2-bade18e055e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:42.347821 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f73bd-b262-4ce3-88e2-bade18e055e2 None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:42.349143 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f73bd-b262-4ce3-88e2-bade18e055e2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:42.349453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f73bd-b262-4ce3-88e2-bade18e055e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:42.351678 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f73bd-b262-4ce3-88e2-bade18e055e2 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:42.353945 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 293/595] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:42.354842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b2330-6cee-46de-a1f6-6489b6b1835c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:42.374951 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 303/596] 10.222.0.81 () {56 vars in 1139 bytes} [Mon Sep 19 11:10:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:42.375802 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f73bd-b262-4ce3-88e2-bade18e055e2 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:42.381692 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:42.415483 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:42.488993 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b2330-6cee-46de-a1f6-6489b6b1835c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:42.517689 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f73bd-b262-4ce3-88e2-bade18e055e2 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:42.535296 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55eb4d4-f50d-47c2-8762-286e27270f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:42.536330 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55eb4d4-f50d-47c2-8762-286e27270f50 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:42.536623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55eb4d4-f50d-47c2-8762-286e27270f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:42.536902 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55eb4d4-f50d-47c2-8762-286e27270f50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:42.546943 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55eb4d4-f50d-47c2-8762-286e27270f50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:42.548962 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55eb4d4-f50d-47c2-8762-286e27270f50 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:42.554775 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa05ac6-5493-47e8-acdb-1e93f417b9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:42.555470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa05ac6-5493-47e8-acdb-1e93f417b9fc None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:42.555967 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa05ac6-5493-47e8-acdb-1e93f417b9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:42.556308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa05ac6-5493-47e8-acdb-1e93f417b9fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:42.563111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-d55eb4d4-f50d-47c2-8762-286e27270f50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:42.563111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:42.563111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:42.563111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:42.563111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:42.563111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:42.563111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:42.563111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:42.563111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:42.563111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:42.563111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:42.563111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:42.563111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:10:42.563111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:10:42.563111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:10:42.564833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:10:42.564833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:42.564833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:42.564833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:42.564833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:42.564833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:42.564833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:42.564833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:42.564833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:42.564833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:42.564833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:42.564833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:42.564833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:42.564833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:42.564833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:42.564833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:42.564833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:42.566459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:10:42.566459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:10:42.566459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:10:42.566459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:10:42.566459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:10:42.566459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:10:42.566459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:42.566459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:42.567204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa05ac6-5493-47e8-acdb-1e93f417b9fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:42.567314 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 294/597] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:42.567314 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55eb4d4-f50d-47c2-8762-286e27270f50 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:42.570650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa05ac6-5493-47e8-acdb-1e93f417b9fc None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:42.584032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-4fa05ac6-5493-47e8-acdb-1e93f417b9fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:42.584032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:42.584032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:42.584032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:42.584032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:42.584032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:42.584032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:42.584032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:42.584032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:42.584032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:42.584032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:42.584032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:42.584032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:10:42.584032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:10:42.584032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:10:42.585131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:10:42.585131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:42.585131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:42.585131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:42.585131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:42.585131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:42.585131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:42.585131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:42.585131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:42.585131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:42.585131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:42.585131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:42.585131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:42.585131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:42.585131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:42.585131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:42.585131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:42.586856 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:10:42.586856 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:10:42.586856 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:10:42.586856 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:10:42.586856 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:10:42.586856 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:10:42.586856 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:42.586856 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:42.587793 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 304/598] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:42.589816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa05ac6-5493-47e8-acdb-1e93f417b9fc None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:42.670963 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:42.689714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:42.699129 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55eb4d4-f50d-47c2-8762-286e27270f50 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:42.710591 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa05ac6-5493-47e8-acdb-1e93f417b9fc None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:42.756231 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd8df4-29b8-44f2-88ee-32577012ab62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:42.757193 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd8df4-29b8-44f2-88ee-32577012ab62 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:42.757482 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd8df4-29b8-44f2-88ee-32577012ab62 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:42.757761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd8df4-29b8-44f2-88ee-32577012ab62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:42.763609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b6432-650b-42f8-81fd-1efb0a535619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:42.764497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b6432-650b-42f8-81fd-1efb0a535619 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:42.764998 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b6432-650b-42f8-81fd-1efb0a535619 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:42.765492 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b6432-650b-42f8-81fd-1efb0a535619 None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:42.766398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b6432-650b-42f8-81fd-1efb0a535619 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:42.766937 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b6432-650b-42f8-81fd-1efb0a535619 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:42.768659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b6432-650b-42f8-81fd-1efb0a535619 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:42.772112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd8df4-29b8-44f2-88ee-32577012ab62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:42.773572 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd8df4-29b8-44f2-88ee-32577012ab62 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:42.787921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 305/599] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:10:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:42.789144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b6432-650b-42f8-81fd-1efb0a535619 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:42.790830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-75fd8df4-29b8-44f2-88ee-32577012ab62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:42.790830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:42.790830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:42.790830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:42.790830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:42.790830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:42.790830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:42.790830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:42.790830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:42.790830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:42.790830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:42.790830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:42.790830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:42.790830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:42.790830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:42.790830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:42.791821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:42.791821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:42.791821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:42.791821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:42.791821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:42.791821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:42.791821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:42.791821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:42.791821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:42.791821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:42.791821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:42.791821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:42.791821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:42.791821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:42.791821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:42.791821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:42.791821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:42.792754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:42.792754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:42.792754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:42.792754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 295/600] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:42.793005 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd8df4-29b8-44f2-88ee-32577012ab62 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:42.902270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b6432-650b-42f8-81fd-1efb0a535619 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:42.919851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd8df4-29b8-44f2-88ee-32577012ab62 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:42.938711 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2dab09-a44b-406c-8c28-341a3c9c4cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:42.939642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dab09-a44b-406c-8c28-341a3c9c4cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:42.940174 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dab09-a44b-406c-8c28-341a3c9c4cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:42.940692 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dab09-a44b-406c-8c28-341a3c9c4cd0 None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:42.941588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2dab09-a44b-406c-8c28-341a3c9c4cd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:42.942123 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2dab09-a44b-406c-8c28-341a3c9c4cd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:42.943748 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2dab09-a44b-406c-8c28-341a3c9c4cd0 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:42.950979 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:42.957723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 306/601] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:10:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:42.958815 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4b508-3b07-4c48-b3e8-8e991b4ee994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:42.959146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2dab09-a44b-406c-8c28-341a3c9c4cd0 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:42.959475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4b508-3b07-4c48-b3e8-8e991b4ee994 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:42.959785 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4b508-3b07-4c48-b3e8-8e991b4ee994 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:42.960064 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4b508-3b07-4c48-b3e8-8e991b4ee994 None admin] PATH_INFO: `/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/f9f494df37254102b339df35a151e075/roles/af9d39307d714527b14b2045ab466e7a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:42.963460 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:42.979511 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4b508-3b07-4c48-b3e8-8e991b4ee994 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed9a0b626e3e4b93b391158e981ec586, user_id=f9f494df37254102b339df35a151e075, role_id=af9d39307d714527b14b2045ab466e7a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:42.981152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4b508-3b07-4c48-b3e8-8e991b4ee994 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:43.000880 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 296/602] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:10:42 2022] PUT /identity/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/f9f494df37254102b339df35a151e075/roles/af9d39307d714527b14b2045ab466e7a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:10:43.000880 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4b508-3b07-4c48-b3e8-8e991b4ee994 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:43.110770 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4b508-3b07-4c48-b3e8-8e991b4ee994 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:43.125676 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2dab09-a44b-406c-8c28-341a3c9c4cd0 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:43.151206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfd3a3-1446-4018-894f-6fa40a9b2ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:43.152057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfd3a3-1446-4018-894f-6fa40a9b2ed5 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:43.152409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfd3a3-1446-4018-894f-6fa40a9b2ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:43.152675 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfd3a3-1446-4018-894f-6fa40a9b2ed5 None admin] PATH_INFO: `/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/88670b0e1a0f460d8f74917c92db9b9a/roles/af9d39307d714527b14b2045ab466e7a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:43.178649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcfd3a3-1446-4018-894f-6fa40a9b2ed5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed9a0b626e3e4b93b391158e981ec586, user_id=88670b0e1a0f460d8f74917c92db9b9a, role_id=af9d39307d714527b14b2045ab466e7a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:43.182524 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcfd3a3-1446-4018-894f-6fa40a9b2ed5 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:43.191921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ba4c0-fa16-4138-b9c3-6851d5a50187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:43.193029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ba4c0-fa16-4138-b9c3-6851d5a50187 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:43.193578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ba4c0-fa16-4138-b9c3-6851d5a50187 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:43.194082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ba4c0-fa16-4138-b9c3-6851d5a50187 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:43.195078 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ba4c0-fa16-4138-b9c3-6851d5a50187 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:43.195686 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ba4c0-fa16-4138-b9c3-6851d5a50187 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:43.198030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ba4c0-fa16-4138-b9c3-6851d5a50187 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:43.214761 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 307/603] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:43.220961 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ba4c0-fa16-4138-b9c3-6851d5a50187 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:43.233309 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 297/604] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:10:42 2022] PUT /identity/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/88670b0e1a0f460d8f74917c92db9b9a/roles/af9d39307d714527b14b2045ab466e7a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:10:43.321982 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:43.370550 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ba4c0-fa16-4138-b9c3-6851d5a50187 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:43.410271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74a008-ab58-45f0-9ee5-6f7d80deef8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:43.410888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74a008-ab58-45f0-9ee5-6f7d80deef8f None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:43.411175 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74a008-ab58-45f0-9ee5-6f7d80deef8f None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:43.411448 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74a008-ab58-45f0-9ee5-6f7d80deef8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:43.419986 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74a008-ab58-45f0-9ee5-6f7d80deef8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:43.421596 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74a008-ab58-45f0-9ee5-6f7d80deef8f None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:43.434089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-5d74a008-ab58-45f0-9ee5-6f7d80deef8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:43.434089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:43.434089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:43.434089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:43.434089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:43.434089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:43.434089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:43.434089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:43.434089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:43.434089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:43.434089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:43.434089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:43.434089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:10:43.434089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:10:43.434089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:10:43.435033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:10:43.435033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:43.435033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:43.435033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:43.435033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:43.435033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:43.435033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:43.435033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:43.435033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:43.435033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:43.435033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:43.435033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:43.435033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:43.435033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:43.435033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:43.435033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:43.435033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:43.436348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:10:43.436348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:10:43.436348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:10:43.436348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:10:43.436348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:10:43.436348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:10:43.436348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:43.436348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:43.436348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 308/605] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:43.442305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfd3a3-1446-4018-894f-6fa40a9b2ed5 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:43.481436 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:43.574228 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfd3a3-1446-4018-894f-6fa40a9b2ed5 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:43.620213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76d58a-7228-4193-955a-3c316915bd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:43.621225 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76d58a-7228-4193-955a-3c316915bd30 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:43.621770 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76d58a-7228-4193-955a-3c316915bd30 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:43.622265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76d58a-7228-4193-955a-3c316915bd30 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:43.623161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76d58a-7228-4193-955a-3c316915bd30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:43.623676 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76d58a-7228-4193-955a-3c316915bd30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:43.625327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76d58a-7228-4193-955a-3c316915bd30 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:43.637923 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 298/606] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:10:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:43.643365 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74a008-ab58-45f0-9ee5-6f7d80deef8f None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:43.696626 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c8001-00c5-428d-8c86-d5e7cf67e067 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:43.697011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c8001-00c5-428d-8c86-d5e7cf67e067 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:43.697200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c8001-00c5-428d-8c86-d5e7cf67e067 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:43.698757 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 299/607] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:10:43.704306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c395d-d94f-49f3-9181-e0a2dafac4fd None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:43.704586 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c395d-d94f-49f3-9181-e0a2dafac4fd None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:43.704859 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c395d-d94f-49f3-9181-e0a2dafac4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:43.724417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f89c395d-d94f-49f3-9181-e0a2dafac4fd None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:43.725656 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f89c395d-d94f-49f3-9181-e0a2dafac4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:43.835202 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74a008-ab58-45f0-9ee5-6f7d80deef8f None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:43.835908 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 300/608] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:10:43.876848 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:43.885653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77d97c-736e-45e2-a5aa-ee99963839c3 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:43.885942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77d97c-736e-45e2-a5aa-ee99963839c3 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:43.886221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77d97c-736e-45e2-a5aa-ee99963839c3 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:43.888533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 301/609] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:10:43.893609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566e009-09ab-4ba1-b5c2-818360935b4d None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:43.893899 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566e009-09ab-4ba1-b5c2-818360935b4d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:43.894187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566e009-09ab-4ba1-b5c2-818360935b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:43.901576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b8c9f-e450-4f47-a91a-d16736c65d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:43.902623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b8c9f-e450-4f47-a91a-d16736c65d7e None admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:43.903599 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b8c9f-e450-4f47-a91a-d16736c65d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:43.904175 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b8c9f-e450-4f47-a91a-d16736c65d7e None admin] PATH_INFO: `/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/65de2e5f67dd4bb6a20cf3934afc8abd/roles/af9d39307d714527b14b2045ab466e7a` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:43.914689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-4566e009-09ab-4ba1-b5c2-818360935b4d None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:43.916093 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4566e009-09ab-4ba1-b5c2-818360935b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:43.933559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b8c9f-e450-4f47-a91a-d16736c65d7e None admin] RBAC: Authorizing `identity:create_grant(project_id=ed9a0b626e3e4b93b391158e981ec586, user_id=65de2e5f67dd4bb6a20cf3934afc8abd, role_id=af9d39307d714527b14b2045ab466e7a)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:43.934932 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b8c9f-e450-4f47-a91a-d16736c65d7e None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:43.941196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:43.959704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 309/610] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:10:43 2022] PUT /identity/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/65de2e5f67dd4bb6a20cf3934afc8abd/roles/af9d39307d714527b14b2045ab466e7a => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:10:44.021715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 302/611] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:10:44.116734 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b8c9f-e450-4f47-a91a-d16736c65d7e None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:44.233457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b8c9f-e450-4f47-a91a-d16736c65d7e None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:44.275007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193555f-e708-418e-92af-2bb7eb9c89a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:44.275651 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193555f-e708-418e-92af-2bb7eb9c89a2 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:44.275959 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193555f-e708-418e-92af-2bb7eb9c89a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:44.276234 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193555f-e708-418e-92af-2bb7eb9c89a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:44.287293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193555f-e708-418e-92af-2bb7eb9c89a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:44.292716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-2193555f-e708-418e-92af-2bb7eb9c89a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:44.292716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:44.292716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:44.292716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:44.292716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:44.292716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:44.292716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:44.292716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:44.292716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:44.292716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:44.292716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:44.292716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:44.292716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:44.292716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:44.292716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:44.292716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:44.293647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:44.293647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:44.293647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:44.293647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:44.293647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:44.293647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:44.293647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:44.293647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:44.293647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:44.293647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:44.293647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:44.293647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:44.293647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:44.293647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:44.293647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:44.293647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:44.295076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:44.295076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:44.295076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:44.295076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:44.295076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:44.295076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:44.295076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:44.295076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:44.295076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:44.295076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:44.295076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:44.295076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:44.295076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 310/612] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:10:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:44.299595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566e009-09ab-4ba1-b5c2-818360935b4d None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:44.373308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193555f-e708-418e-92af-2bb7eb9c89a2 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:44.429712 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566e009-09ab-4ba1-b5c2-818360935b4d None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:44.474810 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426f415-2e30-459d-98fe-f8509a0ca1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:44.475413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426f415-2e30-459d-98fe-f8509a0ca1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:44.475693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426f415-2e30-459d-98fe-f8509a0ca1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:44.475992 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426f415-2e30-459d-98fe-f8509a0ca1b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:44.476164 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:44.476666 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426f415-2e30-459d-98fe-f8509a0ca1b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:44.476969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426f415-2e30-459d-98fe-f8509a0ca1b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:44.484141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426f415-2e30-459d-98fe-f8509a0ca1b5 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:44.505233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 303/613] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:10:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:44.511623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426f415-2e30-459d-98fe-f8509a0ca1b5 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:44.537650 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:44.634560 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193555f-e708-418e-92af-2bb7eb9c89a2 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:44.700072 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426f415-2e30-459d-98fe-f8509a0ca1b5 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:44.703907 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee7422-6df2-4485-ba71-ec354b8d79f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:44.704548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee7422-6df2-4485-ba71-ec354b8d79f9 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:44.704828 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee7422-6df2-4485-ba71-ec354b8d79f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:44.705098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee7422-6df2-4485-ba71-ec354b8d79f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:44.724115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee7422-6df2-4485-ba71-ec354b8d79f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:44.732000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-6dee7422-6df2-4485-ba71-ec354b8d79f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:44.732000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:44.732000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:44.732000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:44.732000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:44.732000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:44.732000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:44.732000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:44.732000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:44.732000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:44.732000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:44.732000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:44.732000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:44.732000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:44.732000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:44.732000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:44.733237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:44.733237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:44.733237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:44.733237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:44.733237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:44.733237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:44.733237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:44.733237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:44.733237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:44.733237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:44.733237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:44.733237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:44.733237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:44.733237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:44.733237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:44.733237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:44.734071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:44.734071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:44.734071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:44.734071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:44.734071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:44.734071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:44.734071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:44.734071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:44.734071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:44.734071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:44.734071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:44.734071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:44.739823 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 311/614] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:10:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:44.745553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee7422-6df2-4485-ba71-ec354b8d79f9 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:44.764561 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83960205-b819-4dc5-b73b-a1ab598d2c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:44.765201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83960205-b819-4dc5-b73b-a1ab598d2c11 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:44.765482 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83960205-b819-4dc5-b73b-a1ab598d2c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:44.765758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83960205-b819-4dc5-b73b-a1ab598d2c11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:44.787508 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83960205-b819-4dc5-b73b-a1ab598d2c11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:44.789103 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83960205-b819-4dc5-b73b-a1ab598d2c11 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:44.818188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-83960205-b819-4dc5-b73b-a1ab598d2c11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:44.818188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:44.818188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:44.818188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:44.818188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:44.818188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:44.818188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:44.818188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:44.818188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:44.818188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:44.818188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:44.818188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:44.818188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:44.818188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:44.818188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:44.818188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:44.820392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:44.820392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:44.820392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:44.820392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:44.820392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:44.820392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:44.820392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:44.820392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:44.820392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:44.820392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:44.820392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:44.820392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:44.820392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:44.820392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:44.820392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:44.820392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:44.820392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:44.822507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:44.822507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:44.822507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:44.822507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 304/615] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:44.828438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83960205-b819-4dc5-b73b-a1ab598d2c11 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:44.881146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:44.893572 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee7422-6df2-4485-ba71-ec354b8d79f9 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:44.931226 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd2977-4d34-4dfd-9deb-26f0f4f404c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:44.932399 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd2977-4d34-4dfd-9deb-26f0f4f404c7 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:44.932645 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd2977-4d34-4dfd-9deb-26f0f4f404c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:44.932898 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd2977-4d34-4dfd-9deb-26f0f4f404c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:44.933567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd2977-4d34-4dfd-9deb-26f0f4f404c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:44.933869 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd2977-4d34-4dfd-9deb-26f0f4f404c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:44.935241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd2977-4d34-4dfd-9deb-26f0f4f404c7 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:44.944224 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 312/616] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:10:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:44.949582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd2977-4d34-4dfd-9deb-26f0f4f404c7 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:44.968241 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:45.038505 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83960205-b819-4dc5-b73b-a1ab598d2c11 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:45.110390 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409257f-56e0-4369-9047-b61ec9a17c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:45.110998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409257f-56e0-4369-9047-b61ec9a17c02 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:45.111332 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409257f-56e0-4369-9047-b61ec9a17c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:45.111611 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409257f-56e0-4369-9047-b61ec9a17c02 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:45.112279 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0409257f-56e0-4369-9047-b61ec9a17c02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:45.112582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0409257f-56e0-4369-9047-b61ec9a17c02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:45.113958 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0409257f-56e0-4369-9047-b61ec9a17c02 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:45.135962 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 305/617] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:45.137935 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd2977-4d34-4dfd-9deb-26f0f4f404c7 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:45.140779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409257f-56e0-4369-9047-b61ec9a17c02 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:45.202684 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73473cb9-5077-4836-a99e-1386fe73469b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:45.203313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73473cb9-5077-4836-a99e-1386fe73469b None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:45.204106 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73473cb9-5077-4836-a99e-1386fe73469b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:45.204390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73473cb9-5077-4836-a99e-1386fe73469b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:45.231531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73473cb9-5077-4836-a99e-1386fe73469b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:45.234391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73473cb9-5077-4836-a99e-1386fe73469b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:45.264780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-73473cb9-5077-4836-a99e-1386fe73469b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:45.264780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:45.264780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:45.264780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:45.264780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:45.264780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:45.264780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:45.264780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:45.264780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:45.264780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:45.264780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:45.264780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:45.264780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:45.264780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:45.264780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:45.264780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:45.265766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:45.265766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:45.265766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:45.265766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:45.265766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:45.265766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:45.265766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:45.265766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:45.265766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:45.265766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:45.265766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:45.265766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:45.265766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:45.265766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:45.265766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:45.265766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:45.265766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:45.266647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:45.266647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:45.266647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:45.266647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 313/618] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:10:45.272978 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73473cb9-5077-4836-a99e-1386fe73469b None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:45.291001 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:45.307227 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409257f-56e0-4369-9047-b61ec9a17c02 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:45.343040 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a80a7-2a3a-4d0a-9988-c2b413ed5544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:45.343800 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a80a7-2a3a-4d0a-9988-c2b413ed5544 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:45.344088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a80a7-2a3a-4d0a-9988-c2b413ed5544 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:45.344418 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a80a7-2a3a-4d0a-9988-c2b413ed5544 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:45.351613 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a80a7-2a3a-4d0a-9988-c2b413ed5544 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:45.353818 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a80a7-2a3a-4d0a-9988-c2b413ed5544 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:45.365913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-4f6a80a7-2a3a-4d0a-9988-c2b413ed5544 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:10:45.365913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:45.365913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:45.365913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:45.365913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:45.365913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:45.365913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:45.365913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:45.365913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:45.365913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:45.365913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:45.365913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:45.365913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:10:45.365913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:10:45.365913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:10:45.365913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:10:45.367057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:45.367057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:45.367057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:10:45.367057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:45.367057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:10:45.367057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:45.367057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:45.367057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:45.367057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:45.367057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:45.367057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:45.367057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:45.367057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:45.367057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:45.367057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:45.367057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:45.367057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:45.367057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:45.368111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:45.368111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:45.368111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:10:45.368111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:10:45.368111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:10:45.368111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:10:45.368111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:10:45.368111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:10:45.368111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:10:45.368111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:45.368111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 306/619] 10.222.0.81 () {56 vars in 1093 bytes} [Mon Sep 19 11:10:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:10:45.368111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:45.380344 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a80a7-2a3a-4d0a-9988-c2b413ed5544 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:45.449299 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73473cb9-5077-4836-a99e-1386fe73469b None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:45.541348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cffd69-0158-4170-901e-03e301a6d9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:45.541884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cffd69-0158-4170-901e-03e301a6d9ff None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:45.542165 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cffd69-0158-4170-901e-03e301a6d9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:45.542437 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cffd69-0158-4170-901e-03e301a6d9ff None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:45.543091 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cffd69-0158-4170-901e-03e301a6d9ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:45.543395 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cffd69-0158-4170-901e-03e301a6d9ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:45.544781 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cffd69-0158-4170-901e-03e301a6d9ff None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:45.595217 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 314/620] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:45.604995 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cffd69-0158-4170-901e-03e301a6d9ff None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:45.714349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a80a7-2a3a-4d0a-9988-c2b413ed5544 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:45.804219 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:45.832176 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3ca86-2a27-4dcf-a2da-bea40aea9dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:45.832796 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3ca86-2a27-4dcf-a2da-bea40aea9dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:45.835784 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3ca86-2a27-4dcf-a2da-bea40aea9dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:45.835784 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3ca86-2a27-4dcf-a2da-bea40aea9dc0 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:45.835784 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3ca86-2a27-4dcf-a2da-bea40aea9dc0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:45.835784 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3ca86-2a27-4dcf-a2da-bea40aea9dc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:45.850926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3ca86-2a27-4dcf-a2da-bea40aea9dc0 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:45.879011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 307/621] 10.222.0.81 () {56 vars in 1133 bytes} [Mon Sep 19 11:10:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 499 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:45.896173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3ca86-2a27-4dcf-a2da-bea40aea9dc0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:45.897870 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cffd69-0158-4170-901e-03e301a6d9ff None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:45.963795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78531c-1cfc-4799-8e8e-d403f9f86c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:45.963795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78531c-1cfc-4799-8e8e-d403f9f86c2b None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:45.964558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78531c-1cfc-4799-8e8e-d403f9f86c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:45.964558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78531c-1cfc-4799-8e8e-d403f9f86c2b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:45.982643 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78531c-1cfc-4799-8e8e-d403f9f86c2b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:45.984132 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78531c-1cfc-4799-8e8e-d403f9f86c2b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:46.001783 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:46.014513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-0d78531c-1cfc-4799-8e8e-d403f9f86c2b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:10:46.014513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:46.014513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:46.014513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:46.014513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:46.014513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:46.014513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:46.014513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:46.014513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:46.014513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:46.014513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:46.014513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:46.014513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:10:46.014513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:10:46.014513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:10:46.014513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:10:46.015519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:46.015519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:46.015519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:10:46.015519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:46.015519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:10:46.015519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:46.015519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:46.015519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:46.015519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:46.015519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:46.015519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:46.015519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:46.015519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:46.015519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:46.015519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:46.015519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:46.015519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:46.015519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:46.016523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:46.016523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:46.016523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:10:46.016523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:10:46.016523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:10:46.016523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:10:46.016523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:10:46.016523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:10:46.016523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:10:46.016523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:46.016523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 315/622] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:10:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:46.017642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54def132-c90c-4540-a724-4d701aa4d10f None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:46.017816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54def132-c90c-4540-a724-4d701aa4d10f None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:46.019548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54def132-c90c-4540-a724-4d701aa4d10f None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:46.019548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 316/623] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:46.026460 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54def132-c90c-4540-a724-4d701aa4d10f None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:46.089527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3ca86-2a27-4dcf-a2da-bea40aea9dc0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:46.124379 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0993a-e4b3-4d7f-a7c2-eafe29922548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:46.124999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0993a-e4b3-4d7f-a7c2-eafe29922548 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:46.125274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0993a-e4b3-4d7f-a7c2-eafe29922548 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:46.125547 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0993a-e4b3-4d7f-a7c2-eafe29922548 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:46.142253 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54def132-c90c-4540-a724-4d701aa4d10f None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:46.168310 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:46.178700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d4090-170e-4f97-85a1-d18118f96cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:46.179332 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d4090-170e-4f97-85a1-d18118f96cfa None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:46.179609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d4090-170e-4f97-85a1-d18118f96cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:46.179907 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d4090-170e-4f97-85a1-d18118f96cfa None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:46.180676 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d4090-170e-4f97-85a1-d18118f96cfa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:46.180981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d4090-170e-4f97-85a1-d18118f96cfa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:46.182387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d4090-170e-4f97-85a1-d18118f96cfa None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:46.199287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0993a-e4b3-4d7f-a7c2-eafe29922548 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:46.205653 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 317/624] 10.222.0.81 () {56 vars in 1135 bytes} [Mon Sep 19 11:10:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:46.205811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0993a-e4b3-4d7f-a7c2-eafe29922548 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:46.205927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b4d60-e1a3-4dab-b9c9-33984ccbad7e None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:46.205927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b4d60-e1a3-4dab-b9c9-33984ccbad7e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:46.205927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b4d60-e1a3-4dab-b9c9-33984ccbad7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:46.220895 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-981b4d60-e1a3-4dab-b9c9-33984ccbad7e None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:46.222066 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-981b4d60-e1a3-4dab-b9c9-33984ccbad7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:46.247972 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:46.299921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 318/625] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:46.301227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a0eb8-bd01-4343-adf5-994b538bab11 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:46.301512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a0eb8-bd01-4343-adf5-994b538bab11 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:46.301780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a0eb8-bd01-4343-adf5-994b538bab11 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:46.306843 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 319/626] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:46.306843 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a0eb8-bd01-4343-adf5-994b538bab11 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:46.327800 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 308/627] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 11:10:46.330152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7649c-17a6-45be-b074-886dc42d0491 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:46.330434 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7649c-17a6-45be-b074-886dc42d0491 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:46.330729 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7649c-17a6-45be-b074-886dc42d0491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:46.350006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-85e7649c-17a6-45be-b074-886dc42d0491 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:46.351802 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-85e7649c-17a6-45be-b074-886dc42d0491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:46.430375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 309/628] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:46.431308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a0eb8-bd01-4343-adf5-994b538bab11 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:46.432417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7649c-17a6-45be-b074-886dc42d0491 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:46.448616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:46.468902 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c3514-6197-4aa7-bf3f-e392ad56d9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:46.469679 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c3514-6197-4aa7-bf3f-e392ad56d9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:46.470374 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c3514-6197-4aa7-bf3f-e392ad56d9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:46.470495 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c3514-6197-4aa7-bf3f-e392ad56d9c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:46.481917 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c3514-6197-4aa7-bf3f-e392ad56d9c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:46.483804 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c3514-6197-4aa7-bf3f-e392ad56d9c5 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:46.510702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-056c3514-6197-4aa7-bf3f-e392ad56d9c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:46.510702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:46.510702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:46.510702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:46.510702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:46.510702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:46.510702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:46.510702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:46.510702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:46.510702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:46.510702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:46.510702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:46.510702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:46.510702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:46.510702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:46.510702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:46.511717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:46.511717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:46.511717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:46.511717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:46.511717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:46.511717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:46.511717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:46.511717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:46.511717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:46.511717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:46.511717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:46.511717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:46.511717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:46.511717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:46.511717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:46.511717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:46.511717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:46.512650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:46.512650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:46.512650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:46.512650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 320/629] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:46.513346 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ff166-7b8b-4268-8d59-e173e8e632b4 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:46.515787 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ff166-7b8b-4268-8d59-e173e8e632b4 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:46.515787 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ff166-7b8b-4268-8d59-e173e8e632b4 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:46.515787 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 321/630] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:46.520075 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bee32-a7bd-4f47-858f-0eb309023ae1 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:46.520357 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bee32-a7bd-4f47-858f-0eb309023ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:46.520626 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bee32-a7bd-4f47-858f-0eb309023ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:46.539616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-843bee32-a7bd-4f47-858f-0eb309023ae1 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:46.540866 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-843bee32-a7bd-4f47-858f-0eb309023ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:46.559391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:46.597031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7649c-17a6-45be-b074-886dc42d0491 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:46.612129 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 322/631] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:10:46.612709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843bee32-a7bd-4f47-858f-0eb309023ae1 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:46.646099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9646d-ca37-464d-b4e8-066eb4a1e507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:46.647751 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9646d-ca37-464d-b4e8-066eb4a1e507 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:46.648278 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9646d-ca37-464d-b4e8-066eb4a1e507 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:46.648746 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9646d-ca37-464d-b4e8-066eb4a1e507 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:46.661320 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9646d-ca37-464d-b4e8-066eb4a1e507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:46.663630 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9646d-ca37-464d-b4e8-066eb4a1e507 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:46.683750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-37e9646d-ca37-464d-b4e8-066eb4a1e507 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:46.683750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:46.683750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:46.683750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:46.683750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:46.683750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:46.683750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:46.683750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:46.683750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:46.683750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:46.683750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:46.683750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:46.683750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:46.683750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:46.683750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:46.683750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:46.684740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:46.684740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:46.684740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:46.684740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:46.684740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:46.684740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:46.684740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:46.684740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:46.684740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:46.684740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:46.684740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:46.684740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:46.684740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:46.684740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:46.684740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:46.684740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:46.684740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:46.685622 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:46.685622 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:46.685622 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:46.687957 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 310/632] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:46.688947 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9646d-ca37-464d-b4e8-066eb4a1e507 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:46.724240 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:46.728308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843bee32-a7bd-4f47-858f-0eb309023ae1 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:46.764363 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fb1fd-0e7b-463e-894a-924a6451f5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:46.765078 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fb1fd-0e7b-463e-894a-924a6451f5bb None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:46.765428 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fb1fd-0e7b-463e-894a-924a6451f5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:46.765771 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fb1fd-0e7b-463e-894a-924a6451f5bb None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:46.766484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713fb1fd-0e7b-463e-894a-924a6451f5bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:46.766852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713fb1fd-0e7b-463e-894a-924a6451f5bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:46.768264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713fb1fd-0e7b-463e-894a-924a6451f5bb None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:46.785343 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 323/633] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:46.785343 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fb1fd-0e7b-463e-894a-924a6451f5bb None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:46.798624 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9646d-ca37-464d-b4e8-066eb4a1e507 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:46.826379 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:46.838701 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d0094-f222-4255-a56c-4595181b431c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:46.839363 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d0094-f222-4255-a56c-4595181b431c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:46.839645 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d0094-f222-4255-a56c-4595181b431c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:46.839964 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d0094-f222-4255-a56c-4595181b431c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:46.851749 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d0094-f222-4255-a56c-4595181b431c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:46.858608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-026d0094-f222-4255-a56c-4595181b431c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:46.858608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:46.858608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:46.858608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:46.858608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:46.858608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:46.858608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:46.858608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:46.858608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:46.858608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:46.858608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:46.858608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:46.858608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:46.858608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:46.858608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:46.858608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:46.859536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:46.859536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:46.859536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:46.859536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:46.859536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:46.859536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:46.859536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:46.859536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:46.859536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:46.859536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:46.859536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:46.859536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:46.859536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:46.859536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:46.859536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:46.859536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:46.860415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:46.860415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:46.860415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:46.860415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:46.860415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:46.860415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:46.860415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:46.860415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:46.860415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:46.860415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:46.860415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:46.860415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:46.866315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 311/634] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:10:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:46.866315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d0094-f222-4255-a56c-4595181b431c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:46.890976 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fb1fd-0e7b-463e-894a-924a6451f5bb None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:46.925080 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dae7c4-7fa4-4aa6-852b-1eb5b20263ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:46.925677 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dae7c4-7fa4-4aa6-852b-1eb5b20263ec None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:46.925958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dae7c4-7fa4-4aa6-852b-1eb5b20263ec None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:46.926229 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dae7c4-7fa4-4aa6-852b-1eb5b20263ec None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:46.926889 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dae7c4-7fa4-4aa6-852b-1eb5b20263ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:46.927190 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dae7c4-7fa4-4aa6-852b-1eb5b20263ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:46.928577 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dae7c4-7fa4-4aa6-852b-1eb5b20263ec None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:46.941903 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 324/635] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:46.942524 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dae7c4-7fa4-4aa6-852b-1eb5b20263ec None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:46.959612 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:46.968121 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d0094-f222-4255-a56c-4595181b431c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:47.002540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac3955-58db-40a4-89b3-195936110d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:47.003166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac3955-58db-40a4-89b3-195936110d03 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:47.003450 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac3955-58db-40a4-89b3-195936110d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:47.003732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac3955-58db-40a4-89b3-195936110d03 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:47.013730 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac3955-58db-40a4-89b3-195936110d03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:47.018982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-63ac3955-58db-40a4-89b3-195936110d03 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:47.018982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:47.018982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:47.018982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:47.018982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:47.018982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:47.018982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:47.018982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:47.018982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:47.018982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:47.018982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:47.018982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:47.018982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:47.018982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:47.018982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:47.018982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:47.019969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:47.019969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:47.019969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:47.019969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:47.019969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:47.019969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:47.019969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:47.019969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:47.019969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:47.019969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:47.019969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:47.019969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:47.019969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:47.019969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:47.019969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:47.019969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:47.020801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:47.020801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:47.020801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:47.020801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:47.020801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:47.020801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:47.020801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:47.020801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:47.020801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:47.020801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:47.020801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:47.020801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:47.020801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 312/636] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:10:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:47.021497 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac3955-58db-40a4-89b3-195936110d03 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:47.045474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dae7c4-7fa4-4aa6-852b-1eb5b20263ec None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:47.065453 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:47.079813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db210158-1631-43c8-a294-85828ccc12d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:47.080438 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db210158-1631-43c8-a294-85828ccc12d5 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:47.080716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db210158-1631-43c8-a294-85828ccc12d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:47.080988 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db210158-1631-43c8-a294-85828ccc12d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:47.088200 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db210158-1631-43c8-a294-85828ccc12d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:47.089617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db210158-1631-43c8-a294-85828ccc12d5 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:47.101271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-db210158-1631-43c8-a294-85828ccc12d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:47.101271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:47.101271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:47.101271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:47.101271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:47.101271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:47.101271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:47.101271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:47.101271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:47.101271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:47.101271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:47.101271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:47.101271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:10:47.101271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:10:47.101271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:10:47.102347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:10:47.102347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:47.102347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:47.102347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:47.102347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:47.102347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:47.102347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:47.102347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:47.102347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:47.102347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:47.102347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:47.102347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:47.102347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:47.102347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:47.102347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:47.102347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:47.102347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:47.103279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:10:47.103279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:10:47.103279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:10:47.103279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:10:47.103279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:10:47.103279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:10:47.103279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:47.103279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:47.103279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 325/637] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:47.103279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db210158-1631-43c8-a294-85828ccc12d5 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:47.121794 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac3955-58db-40a4-89b3-195936110d03 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:47.155240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cae3b-2951-439e-8980-3f2820a5953a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:47.155864 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cae3b-2951-439e-8980-3f2820a5953a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:47.156156 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cae3b-2951-439e-8980-3f2820a5953a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:47.156432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cae3b-2951-439e-8980-3f2820a5953a None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:47.157090 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cae3b-2951-439e-8980-3f2820a5953a None admin] RBAC: Adding query filter params (name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:47.157401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cae3b-2951-439e-8980-3f2820a5953a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:47.158789 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cae3b-2951-439e-8980-3f2820a5953a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:47.166469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 313/638] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:10:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:47.167322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cae3b-2951-439e-8980-3f2820a5953a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:47.185576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:47.210576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db210158-1631-43c8-a294-85828ccc12d5 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:47.244537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95f70c-21e0-4e96-b57a-3056c432d1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:47.245157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95f70c-21e0-4e96-b57a-3056c432d1df None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:47.245431 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95f70c-21e0-4e96-b57a-3056c432d1df None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:47.245698 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95f70c-21e0-4e96-b57a-3056c432d1df None admin] PATH_INFO: `/v3/roles/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:47.255600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95f70c-21e0-4e96-b57a-3056c432d1df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:47.260958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-7e95f70c-21e0-4e96-b57a-3056c432d1df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:47.260958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:47.260958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:47.260958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:47.260958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:47.260958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:47.260958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:47.260958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:47.260958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:47.260958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:47.260958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:47.260958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:47.260958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:47.260958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:47.260958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:47.260958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:47.261908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:47.261908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:47.261908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:47.261908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:47.261908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:47.261908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:47.261908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:47.261908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:47.261908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:47.261908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:47.261908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:47.261908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:47.261908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:47.261908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:47.261908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:47.261908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:47.262760 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:47.262760 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:47.262760 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:47.262760 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:47.262760 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:47.262760 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:47.262760 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:47.262760 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:47.262760 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:47.262760 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:47.262760 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:47.262760 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:47.262760 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 326/639] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:10:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:47.262760 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95f70c-21e0-4e96-b57a-3056c432d1df None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:47.271043 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cae3b-2951-439e-8980-3f2820a5953a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:47.278182 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:47.305418 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849a8b1-030e-4379-843a-2789877ac6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:47.306584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849a8b1-030e-4379-843a-2789877ac6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:47.306864 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849a8b1-030e-4379-843a-2789877ac6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:47.307135 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849a8b1-030e-4379-843a-2789877ac6d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:47.313722 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849a8b1-030e-4379-843a-2789877ac6d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:47.315174 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849a8b1-030e-4379-843a-2789877ac6d3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:47.326318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-e849a8b1-030e-4379-843a-2789877ac6d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:47.326318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:47.326318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:47.326318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:47.326318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:47.326318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:47.326318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:47.326318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:47.326318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:47.326318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:47.326318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:47.326318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:47.326318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:10:47.326318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:10:47.326318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:10:47.327274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:10:47.327274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:47.327274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:47.327274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:47.327274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:47.327274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:47.327274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:47.327274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:47.327274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:47.327274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:47.327274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:47.327274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:47.327274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:47.327274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:47.327274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:47.327274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:47.327274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:47.328935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:10:47.328935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:10:47.328935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:10:47.328935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:10:47.328935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:10:47.328935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:10:47.328935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:47.328935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:47.328935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 314/640] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:47.328935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849a8b1-030e-4379-843a-2789877ac6d3 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:47.363880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95f70c-21e0-4e96-b57a-3056c432d1df None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:47.397307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac370baf-87ae-4fd0-a2c3-778f0df3d207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:47.397835 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac370baf-87ae-4fd0-a2c3-778f0df3d207 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:47.398113 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac370baf-87ae-4fd0-a2c3-778f0df3d207 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:47.398383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac370baf-87ae-4fd0-a2c3-778f0df3d207 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:47.399030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac370baf-87ae-4fd0-a2c3-778f0df3d207 None admin] RBAC: Adding query filter params (name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:47.399402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac370baf-87ae-4fd0-a2c3-778f0df3d207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:47.400898 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac370baf-87ae-4fd0-a2c3-778f0df3d207 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:47.403003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:47.409649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 327/641] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:10:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:47.410446 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac370baf-87ae-4fd0-a2c3-778f0df3d207 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:47.428197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849a8b1-030e-4379-843a-2789877ac6d3 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:47.461821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9797b-ba82-4205-8508-7b4e98e6b16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:47.462433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9797b-ba82-4205-8508-7b4e98e6b16f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:47.462711 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9797b-ba82-4205-8508-7b4e98e6b16f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:47.462991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9797b-ba82-4205-8508-7b4e98e6b16f None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:47.463678 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9797b-ba82-4205-8508-7b4e98e6b16f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:47.464017 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9797b-ba82-4205-8508-7b4e98e6b16f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:47.465461 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9797b-ba82-4205-8508-7b4e98e6b16f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:47.476908 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 315/642] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:10:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:47.477739 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9797b-ba82-4205-8508-7b4e98e6b16f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:47.504901 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:47.508881 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac370baf-87ae-4fd0-a2c3-778f0df3d207 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:47.541859 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320495b-1a61-485e-8215-0487fd7d9d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:47.542480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320495b-1a61-485e-8215-0487fd7d9d60 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:47.542757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320495b-1a61-485e-8215-0487fd7d9d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:47.543030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320495b-1a61-485e-8215-0487fd7d9d60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:47.553249 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320495b-1a61-485e-8215-0487fd7d9d60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:47.554676 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320495b-1a61-485e-8215-0487fd7d9d60 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:47.568888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-9320495b-1a61-485e-8215-0487fd7d9d60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:47.568888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:47.568888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:47.568888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:47.568888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:47.568888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:47.568888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:47.568888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:47.568888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:47.568888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:47.568888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:47.568888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:47.568888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:47.568888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:47.568888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:47.568888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:47.570051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:47.570051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:47.570051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:47.570051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:47.570051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:47.570051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:47.570051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:47.570051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:47.570051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:47.570051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:47.570051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:47.570051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:47.570051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:47.570051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:47.570051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:47.570051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:47.570051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:47.571376 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:47.571376 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:47.571376 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:47.571376 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 328/643] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:47.571376 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320495b-1a61-485e-8215-0487fd7d9d60 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:47.579682 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9797b-ba82-4205-8508-7b4e98e6b16f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:47.613566 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2570fd-3081-47c7-b868-ab64405bd62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:47.614162 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2570fd-3081-47c7-b868-ab64405bd62f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:47.614427 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2570fd-3081-47c7-b868-ab64405bd62f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:47.614691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2570fd-3081-47c7-b868-ab64405bd62f None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:47.615340 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2570fd-3081-47c7-b868-ab64405bd62f None admin] RBAC: Adding query filter params (name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:47.615638 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2570fd-3081-47c7-b868-ab64405bd62f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:47.617030 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2570fd-3081-47c7-b868-ab64405bd62f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:47.620951 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:47.625191 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 316/644] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:10:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:47.625964 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2570fd-3081-47c7-b868-ab64405bd62f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:47.676129 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320495b-1a61-485e-8215-0487fd7d9d60 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:47.708827 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece073d-5dc5-4dd8-81be-93a3322a6999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:47.709418 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece073d-5dc5-4dd8-81be-93a3322a6999 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:47.709689 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece073d-5dc5-4dd8-81be-93a3322a6999 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:47.709959 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece073d-5dc5-4dd8-81be-93a3322a6999 None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:47.710629 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece073d-5dc5-4dd8-81be-93a3322a6999 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:47.710929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece073d-5dc5-4dd8-81be-93a3322a6999 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:47.712450 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece073d-5dc5-4dd8-81be-93a3322a6999 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:47.715957 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:47.724147 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 329/645] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:10:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:47.724735 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece073d-5dc5-4dd8-81be-93a3322a6999 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:47.729260 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2570fd-3081-47c7-b868-ab64405bd62f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:47.762757 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8cf02c-8614-49cb-a794-cd8d81cf3d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:47.763384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cf02c-8614-49cb-a794-cd8d81cf3d7e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:47.763666 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cf02c-8614-49cb-a794-cd8d81cf3d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:47.763973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cf02c-8614-49cb-a794-cd8d81cf3d7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:47.774333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8cf02c-8614-49cb-a794-cd8d81cf3d7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:47.775810 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8cf02c-8614-49cb-a794-cd8d81cf3d7e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:47.789618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-4f8cf02c-8614-49cb-a794-cd8d81cf3d7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:47.789618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:47.789618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:47.789618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:47.789618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:47.789618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:47.789618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:47.789618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:47.789618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:47.789618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:47.789618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:47.789618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:47.789618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:47.789618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:47.789618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:47.789618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:47.791108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:47.791108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:47.791108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:47.791108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:47.791108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:47.791108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:47.791108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:47.791108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:47.791108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:47.791108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:47.791108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:47.791108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:47.791108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:47.791108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:47.791108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:47.791108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:47.791108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:47.792054 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:47.792054 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:47.792054 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:47.792054 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 317/646] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:47.792341 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8cf02c-8614-49cb-a794-cd8d81cf3d7e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:47.828344 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece073d-5dc5-4dd8-81be-93a3322a6999 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:47.839583 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:47.865153 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b5068-07ce-4ea5-926b-593db7b5be62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:47.865809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b5068-07ce-4ea5-926b-593db7b5be62 None admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:47.866090 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b5068-07ce-4ea5-926b-593db7b5be62 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:47.866366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b5068-07ce-4ea5-926b-593db7b5be62 None admin] PATH_INFO: `/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/1c8a9732807d4921a41a6a2d5019e2f5/roles/af9d39307d714527b14b2045ab466e7a` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:47.882120 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b5068-07ce-4ea5-926b-593db7b5be62 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed9a0b626e3e4b93b391158e981ec586, user_id=1c8a9732807d4921a41a6a2d5019e2f5, role_id=af9d39307d714527b14b2045ab466e7a)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:47.883545 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b5068-07ce-4ea5-926b-593db7b5be62 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:47.892226 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8cf02c-8614-49cb-a794-cd8d81cf3d7e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:47.898137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 330/647] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:10:47 2022] PUT /identity/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/1c8a9732807d4921a41a6a2d5019e2f5/roles/af9d39307d714527b14b2045ab466e7a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:10:47.898949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b5068-07ce-4ea5-926b-593db7b5be62 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:47.935089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17fca0e-e927-4df6-b6ae-4ca12d5f8aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:47.935619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fca0e-e927-4df6-b6ae-4ca12d5f8aba None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:47.935971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fca0e-e927-4df6-b6ae-4ca12d5f8aba None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:47.936329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fca0e-e927-4df6-b6ae-4ca12d5f8aba None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:47.936990 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17fca0e-e927-4df6-b6ae-4ca12d5f8aba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:47.937326 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17fca0e-e927-4df6-b6ae-4ca12d5f8aba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:47.938774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17fca0e-e927-4df6-b6ae-4ca12d5f8aba None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:47.949887 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:47.959930 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 318/648] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:47.960966 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17fca0e-e927-4df6-b6ae-4ca12d5f8aba None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:48.016880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b5068-07ce-4ea5-926b-593db7b5be62 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:48.057476 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f3224f-f580-4239-8306-06858c9b3a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:48.058152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3224f-f580-4239-8306-06858c9b3a91 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:48.058435 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3224f-f580-4239-8306-06858c9b3a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:48.058712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3224f-f580-4239-8306-06858c9b3a91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:48.073409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f3224f-f580-4239-8306-06858c9b3a91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:48.074913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f3224f-f580-4239-8306-06858c9b3a91 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:48.079457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17fca0e-e927-4df6-b6ae-4ca12d5f8aba None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:48.094088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-95f3224f-f580-4239-8306-06858c9b3a91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:48.094088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:48.094088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:48.094088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:48.094088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:48.094088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:48.094088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:48.094088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:48.094088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:48.094088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:48.094088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:48.094088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:48.094088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:48.094088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:48.094088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:48.094088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:48.095089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:48.095089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:48.095089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:48.095089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:48.095089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:48.095089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:48.095089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:48.095089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:48.095089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:48.095089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:48.095089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:48.095089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:48.095089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:48.095089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:48.095089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:48.095089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:48.095089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:48.096008 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:48.096008 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:48.096008 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:48.096192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 331/649] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:48.097186 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f3224f-f580-4239-8306-06858c9b3a91 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:48.113333 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:48.118803 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea74247-bc8e-4ed8-be8d-f048070149a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:48.119507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea74247-bc8e-4ed8-be8d-f048070149a0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:48.120459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea74247-bc8e-4ed8-be8d-f048070149a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:48.120738 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea74247-bc8e-4ed8-be8d-f048070149a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:48.121468 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea74247-bc8e-4ed8-be8d-f048070149a0 None admin] RBAC: Adding query filter params (role.id=af9d39307d714527b14b2045ab466e7a, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=5abd70a18ccd407bafe3b61c8870986e) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:48.121783 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea74247-bc8e-4ed8-be8d-f048070149a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:48.123375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea74247-bc8e-4ed8-be8d-f048070149a0 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:48.130115 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 319/650] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:10:47 2022] GET /identity/v3/role_assignments?user.id=5abd70a18ccd407bafe3b61c8870986e&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=af9d39307d714527b14b2045ab466e7a => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:48.130983 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea74247-bc8e-4ed8-be8d-f048070149a0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:48.219578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:48.234054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f3224f-f580-4239-8306-06858c9b3a91 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:48.264053 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea74247-bc8e-4ed8-be8d-f048070149a0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:48.280369 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da054647-8c8c-419c-aa60-76dd4a90b6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:48.280989 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da054647-8c8c-419c-aa60-76dd4a90b6f4 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:48.281772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da054647-8c8c-419c-aa60-76dd4a90b6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:48.282250 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da054647-8c8c-419c-aa60-76dd4a90b6f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:48.284878 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da054647-8c8c-419c-aa60-76dd4a90b6f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:48.285193 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da054647-8c8c-419c-aa60-76dd4a90b6f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:48.286727 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da054647-8c8c-419c-aa60-76dd4a90b6f4 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:48.304106 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 332/651] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:48.305284 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da054647-8c8c-419c-aa60-76dd4a90b6f4 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:48.309307 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63f712-3cc4-466a-973a-7cfc279a5c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:48.310054 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63f712-3cc4-466a-973a-7cfc279a5c7e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:48.310436 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63f712-3cc4-466a-973a-7cfc279a5c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:48.310811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63f712-3cc4-466a-973a-7cfc279a5c7e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:48.319117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63f712-3cc4-466a-973a-7cfc279a5c7e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:48.320862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63f712-3cc4-466a-973a-7cfc279a5c7e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:48.335287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-3c63f712-3cc4-466a-973a-7cfc279a5c7e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 19 11:10:48.335287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:48.335287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:48.335287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:48.335287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:48.335287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:48.335287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:48.335287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:48.335287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:48.335287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:48.335287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:48.335287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:48.335287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:10:48.335287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:10:48.335287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:10:48.335287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:10:48.338562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:48.338562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:48.338562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:10:48.338562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:48.338562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:10:48.338562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:48.338562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:48.338562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:48.338562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:48.338562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:48.338562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:48.338562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:48.338562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:48.338562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:48.338562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:48.338562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:48.338562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:48.338562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:48.340830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:48.340830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:48.340830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:10:48.340830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:10:48.340830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:10:48.340830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:10:48.340830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:10:48.340830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:10:48.340830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 19 11:10:48.340830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:48.340830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 320/652] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:48.344715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63f712-3cc4-466a-973a-7cfc279a5c7e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:48.391957 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:48.434067 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da054647-8c8c-419c-aa60-76dd4a90b6f4 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:48.463747 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63f712-3cc4-466a-973a-7cfc279a5c7e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:48.476083 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66c4d2-5504-46e0-9877-875e3a73d12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:48.477077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66c4d2-5504-46e0-9877-875e3a73d12b None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:48.478102 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66c4d2-5504-46e0-9877-875e3a73d12b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:48.478582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66c4d2-5504-46e0-9877-875e3a73d12b None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:48.479538 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66c4d2-5504-46e0-9877-875e3a73d12b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:48.480051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66c4d2-5504-46e0-9877-875e3a73d12b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:48.481672 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66c4d2-5504-46e0-9877-875e3a73d12b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:48.489589 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:48.496233 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 333/653] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:48.497314 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66c4d2-5504-46e0-9877-875e3a73d12b None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:48.507039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62147831-d7a2-4a56-b3bc-be83dec46887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:48.507868 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62147831-d7a2-4a56-b3bc-be83dec46887 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:48.508151 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62147831-d7a2-4a56-b3bc-be83dec46887 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:48.508476 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62147831-d7a2-4a56-b3bc-be83dec46887 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:48.519191 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62147831-d7a2-4a56-b3bc-be83dec46887 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:48.520661 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62147831-d7a2-4a56-b3bc-be83dec46887 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:48.536081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-62147831-d7a2-4a56-b3bc-be83dec46887 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:10:48.536081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:48.536081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:48.536081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:48.536081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:48.536081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:48.536081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:48.536081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:48.536081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:48.536081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:48.536081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:48.536081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:48.536081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:10:48.536081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:10:48.536081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:10:48.536081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:10:48.537245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:48.537245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:48.537245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:10:48.537245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:48.537245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:10:48.537245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:48.537245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:48.537245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:48.537245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:48.537245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:48.537245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:48.537245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:48.537245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:48.537245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:48.537245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:48.537245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:48.537245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:48.537245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:48.538556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:48.538556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:48.538556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:10:48.538556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:10:48.538556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:10:48.538556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:10:48.538556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:10:48.538556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:10:48.538556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:10:48.538556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:48.538556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 321/654] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:48.538556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62147831-d7a2-4a56-b3bc-be83dec46887 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:48.611048 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66c4d2-5504-46e0-9877-875e3a73d12b None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:48.644100 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:48.652182 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da113aab-7adc-4b17-8bb3-4c6947340511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:48.653149 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da113aab-7adc-4b17-8bb3-4c6947340511 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:48.653868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da113aab-7adc-4b17-8bb3-4c6947340511 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:48.654699 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da113aab-7adc-4b17-8bb3-4c6947340511 None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:48.656614 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da113aab-7adc-4b17-8bb3-4c6947340511 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:48.657197 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da113aab-7adc-4b17-8bb3-4c6947340511 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:48.658961 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da113aab-7adc-4b17-8bb3-4c6947340511 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:48.683827 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 334/655] 10.222.0.81 () {56 vars in 1137 bytes} [Mon Sep 19 11:10:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:48.685642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da113aab-7adc-4b17-8bb3-4c6947340511 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:48.687343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62147831-d7a2-4a56-b3bc-be83dec46887 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:48.733210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0d3ef-80cc-4f9c-999f-84ba28c2ea6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:48.733945 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0d3ef-80cc-4f9c-999f-84ba28c2ea6e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:48.734318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0d3ef-80cc-4f9c-999f-84ba28c2ea6e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:48.734687 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0d3ef-80cc-4f9c-999f-84ba28c2ea6e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:48.745850 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af0d3ef-80cc-4f9c-999f-84ba28c2ea6e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:48.747953 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af0d3ef-80cc-4f9c-999f-84ba28c2ea6e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:48.764523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-6af0d3ef-80cc-4f9c-999f-84ba28c2ea6e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 11:10:48.764523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:48.764523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:48.764523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:48.764523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:48.764523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:48.764523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:48.764523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:48.764523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:48.764523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:48.764523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:48.764523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:48.764523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:10:48.764523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:10:48.764523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:10:48.764523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:10:48.765584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:48.765584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:48.765584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:10:48.765584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:48.765584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:10:48.765584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:48.765584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:48.765584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:48.765584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:48.765584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:48.765584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:48.765584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:48.765584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:48.765584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:48.765584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:48.765584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:48.765584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:48.765584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:48.766531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:48.766531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:48.766531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:10:48.766531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:10:48.766531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:10:48.766531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:10:48.766531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:10:48.766531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:10:48.766531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 11:10:48.766531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:48.767840 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 322/656] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:10:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:48.768831 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0d3ef-80cc-4f9c-999f-84ba28c2ea6e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:48.807084 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da113aab-7adc-4b17-8bb3-4c6947340511 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:48.840425 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:48.850144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470890fa-b36a-43f7-867a-3655c88e83f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:48.851044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470890fa-b36a-43f7-867a-3655c88e83f5 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:48.851555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470890fa-b36a-43f7-867a-3655c88e83f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:48.852111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470890fa-b36a-43f7-867a-3655c88e83f5 None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:48.853309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470890fa-b36a-43f7-867a-3655c88e83f5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:48.854148 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470890fa-b36a-43f7-867a-3655c88e83f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:48.856171 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470890fa-b36a-43f7-867a-3655c88e83f5 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:48.877481 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 335/657] 10.222.0.81 () {56 vars in 1137 bytes} [Mon Sep 19 11:10:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:48.878697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470890fa-b36a-43f7-867a-3655c88e83f5 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:48.910549 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0d3ef-80cc-4f9c-999f-84ba28c2ea6e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:48.939585 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:48.953968 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd377030-44aa-417f-b343-4e85a1e28c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:48.954933 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd377030-44aa-417f-b343-4e85a1e28c14 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:48.955569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd377030-44aa-417f-b343-4e85a1e28c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:48.956073 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd377030-44aa-417f-b343-4e85a1e28c14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:48.998679 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470890fa-b36a-43f7-867a-3655c88e83f5 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:49.035279 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd377030-44aa-417f-b343-4e85a1e28c14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:49.037412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd377030-44aa-417f-b343-4e85a1e28c14 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:49.043716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ecb15-6d0b-4165-a777-4683bc042f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:49.044348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ecb15-6d0b-4165-a777-4683bc042f10 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:49.044622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ecb15-6d0b-4165-a777-4683bc042f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:49.044888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ecb15-6d0b-4165-a777-4683bc042f10 None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:49.045704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ecb15-6d0b-4165-a777-4683bc042f10 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:49.046007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ecb15-6d0b-4165-a777-4683bc042f10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:49.047999 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ecb15-6d0b-4165-a777-4683bc042f10 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:49.074796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 336/658] 10.222.0.81 () {56 vars in 1139 bytes} [Mon Sep 19 11:10:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:49.075707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ecb15-6d0b-4165-a777-4683bc042f10 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:49.089962 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:49.152253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 323/659] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:10:49.153512 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd377030-44aa-417f-b343-4e85a1e28c14 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:49.189734 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ecb15-6d0b-4165-a777-4683bc042f10 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:49.197377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:49.228146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6ce84-3156-4d38-a781-3ab98e8ada78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:49.228768 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6ce84-3156-4d38-a781-3ab98e8ada78 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:49.229042 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6ce84-3156-4d38-a781-3ab98e8ada78 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:49.229311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6ce84-3156-4d38-a781-3ab98e8ada78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:49.270635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd377030-44aa-417f-b343-4e85a1e28c14 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:49.303915 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c6ce84-3156-4d38-a781-3ab98e8ada78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:49.305350 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c6ce84-3156-4d38-a781-3ab98e8ada78 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:49.308633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd567951-7539-4820-bfd0-e532af7a0f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:49.309413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd567951-7539-4820-bfd0-e532af7a0f20 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:49.309845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd567951-7539-4820-bfd0-e532af7a0f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:49.310267 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd567951-7539-4820-bfd0-e532af7a0f20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:49.326252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:49.384646 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd567951-7539-4820-bfd0-e532af7a0f20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:49.386942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd567951-7539-4820-bfd0-e532af7a0f20 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:49.414114 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 337/660] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:10:49.415148 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6ce84-3156-4d38-a781-3ab98e8ada78 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:49.423742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:49.498800 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 324/661] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:10:49.500065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd567951-7539-4820-bfd0-e532af7a0f20 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:49.520211 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6ce84-3156-4d38-a781-3ab98e8ada78 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:49.550077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:49.558795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628baab1-0f89-4748-a966-df38ec38a447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:49.559808 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628baab1-0f89-4748-a966-df38ec38a447 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:49.559808 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628baab1-0f89-4748-a966-df38ec38a447 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:49.560312 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628baab1-0f89-4748-a966-df38ec38a447 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:49.572135 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628baab1-0f89-4748-a966-df38ec38a447 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:49.573914 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628baab1-0f89-4748-a966-df38ec38a447 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:49.590400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-628baab1-0f89-4748-a966-df38ec38a447 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:49.590400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:49.590400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:49.590400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:49.590400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:49.590400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:49.590400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:49.590400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:49.590400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:49.590400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:49.590400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:49.590400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:49.590400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:49.590400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:49.590400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:49.590400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:49.591578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:49.591578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:49.591578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:49.591578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:49.591578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:49.591578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:49.591578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:49.591578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:49.591578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:49.591578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:49.591578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:49.591578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:49.591578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:49.591578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:49.591578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:49.591578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:49.591578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:49.592612 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:49.592612 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:49.592612 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:49.594482 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 338/662] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:49.595405 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628baab1-0f89-4748-a966-df38ec38a447 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:49.615215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd567951-7539-4820-bfd0-e532af7a0f20 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:49.658342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb8db1-fa0a-4114-9ea3-fb3eab6ec3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:49.658988 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb8db1-fa0a-4114-9ea3-fb3eab6ec3a7 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:49.659277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb8db1-fa0a-4114-9ea3-fb3eab6ec3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:49.659553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb8db1-fa0a-4114-9ea3-fb3eab6ec3a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:49.665211 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:49.675032 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb8db1-fa0a-4114-9ea3-fb3eab6ec3a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:49.677172 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb8db1-fa0a-4114-9ea3-fb3eab6ec3a7 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:49.697349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-0fbb8db1-fa0a-4114-9ea3-fb3eab6ec3a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:49.697349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:49.697349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:49.697349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:49.697349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:49.697349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:49.697349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:49.697349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:49.697349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:49.697349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:49.697349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:49.697349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:49.697349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:49.697349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:49.697349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:49.697349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:49.698369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:49.698369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:49.698369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:49.698369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:49.698369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:49.698369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:49.698369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:49.698369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:49.698369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:49.698369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:49.698369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:49.698369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:49.698369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:49.698369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:49.698369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:49.698369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:49.698369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:49.699300 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:49.699300 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:49.699300 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:49.706223 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 325/663] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:49.706223 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb8db1-fa0a-4114-9ea3-fb3eab6ec3a7 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:49.733638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628baab1-0f89-4748-a966-df38ec38a447 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:49.775378 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b29d32-f414-4f55-8cca-b32b98aede30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:49.776457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b29d32-f414-4f55-8cca-b32b98aede30 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:49.776944 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b29d32-f414-4f55-8cca-b32b98aede30 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:49.777419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b29d32-f414-4f55-8cca-b32b98aede30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:49.795652 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b29d32-f414-4f55-8cca-b32b98aede30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:49.797806 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b29d32-f414-4f55-8cca-b32b98aede30 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:49.806664 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:49.817723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-97b29d32-f414-4f55-8cca-b32b98aede30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:49.817723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:49.817723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:49.817723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:49.817723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:49.817723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:49.817723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:49.817723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:49.817723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:49.817723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:49.817723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:49.817723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:49.817723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:49.817723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:49.817723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:49.817723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:49.818714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:49.818714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:49.818714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:49.818714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:49.818714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:49.818714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:49.818714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:49.818714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:49.818714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:49.818714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:49.818714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:49.818714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:49.818714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:49.818714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:49.818714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:49.818714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:49.818714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:49.819633 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:49.819633 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:49.819633 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:49.820714 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb8db1-fa0a-4114-9ea3-fb3eab6ec3a7 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:49.824289 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 339/664] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:49.824523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b29d32-f414-4f55-8cca-b32b98aede30 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:49.868463 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343d694-853c-4545-977a-0fdd82fe2ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:49.869438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343d694-853c-4545-977a-0fdd82fe2ceb None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:49.870295 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343d694-853c-4545-977a-0fdd82fe2ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:49.871055 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343d694-853c-4545-977a-0fdd82fe2ceb None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:49.872011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343d694-853c-4545-977a-0fdd82fe2ceb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:49.872737 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343d694-853c-4545-977a-0fdd82fe2ceb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:49.875097 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343d694-853c-4545-977a-0fdd82fe2ceb None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:49.891823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 326/665] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:49.893249 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343d694-853c-4545-977a-0fdd82fe2ceb None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:49.932661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b29d32-f414-4f55-8cca-b32b98aede30 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:49.948244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:49.974007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be7954-0f2a-44b0-8c12-ce0300cb55a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:49.974602 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be7954-0f2a-44b0-8c12-ce0300cb55a5 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:49.974876 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be7954-0f2a-44b0-8c12-ce0300cb55a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:49.975145 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be7954-0f2a-44b0-8c12-ce0300cb55a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:49.975787 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be7954-0f2a-44b0-8c12-ce0300cb55a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:49.976172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be7954-0f2a-44b0-8c12-ce0300cb55a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:49.978073 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be7954-0f2a-44b0-8c12-ce0300cb55a5 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:49.995031 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 340/666] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:49.995031 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be7954-0f2a-44b0-8c12-ce0300cb55a5 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:50.046466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343d694-853c-4545-977a-0fdd82fe2ceb None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:50.097723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dffb7f-a8c5-46f1-be23-4cfaa56d1ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:50.098236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dffb7f-a8c5-46f1-be23-4cfaa56d1ebe None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:50.098516 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dffb7f-a8c5-46f1-be23-4cfaa56d1ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:50.098785 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dffb7f-a8c5-46f1-be23-4cfaa56d1ebe None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:50.099438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dffb7f-a8c5-46f1-be23-4cfaa56d1ebe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:50.099935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dffb7f-a8c5-46f1-be23-4cfaa56d1ebe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:50.101121 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dffb7f-a8c5-46f1-be23-4cfaa56d1ebe None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:50.107404 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be7954-0f2a-44b0-8c12-ce0300cb55a5 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:50.118302 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 327/667] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:50.119199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dffb7f-a8c5-46f1-be23-4cfaa56d1ebe None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:50.144160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:50.148496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd31210-de18-4152-a0ed-53cca8df09fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:50.149127 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd31210-de18-4152-a0ed-53cca8df09fd None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:50.149968 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd31210-de18-4152-a0ed-53cca8df09fd None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:50.150238 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd31210-de18-4152-a0ed-53cca8df09fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:50.157867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd31210-de18-4152-a0ed-53cca8df09fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:50.159511 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd31210-de18-4152-a0ed-53cca8df09fd None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:50.172329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-2fd31210-de18-4152-a0ed-53cca8df09fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:50.172329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:50.172329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:50.172329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:50.172329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:50.172329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:50.172329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:50.172329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:50.172329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:50.172329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:50.172329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:50.172329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:50.172329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:10:50.172329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:10:50.172329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:10:50.173328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:10:50.173328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:50.173328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:50.173328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:50.173328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:50.173328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:50.173328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:50.173328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:50.173328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:50.173328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:50.173328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:50.173328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:50.173328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:50.173328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:50.173328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:50.173328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:50.173328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:50.174227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:10:50.174227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:10:50.174227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:10:50.174227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:10:50.174227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:10:50.174227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:10:50.174227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:50.174227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:50.174227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 341/668] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:50.174721 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd31210-de18-4152-a0ed-53cca8df09fd None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:50.206289 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:50.247004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dffb7f-a8c5-46f1-be23-4cfaa56d1ebe None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:50.282827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122b74f-5e6f-42a3-9134-017af6ac7488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:50.283509 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b74f-5e6f-42a3-9134-017af6ac7488 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:50.283728 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b74f-5e6f-42a3-9134-017af6ac7488 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:50.284030 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b74f-5e6f-42a3-9134-017af6ac7488 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:50.288402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd31210-de18-4152-a0ed-53cca8df09fd None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:50.291635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122b74f-5e6f-42a3-9134-017af6ac7488 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:50.293127 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122b74f-5e6f-42a3-9134-017af6ac7488 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:50.306254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-b122b74f-5e6f-42a3-9134-017af6ac7488 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:50.306254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:50.306254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:50.306254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:50.306254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:50.306254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:50.306254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:50.306254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:50.306254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:50.306254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:50.306254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:50.306254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:50.306254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:10:50.306254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:10:50.306254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:10:50.307154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:10:50.307154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:50.307154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:50.307154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:50.307154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:50.307154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:50.307154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:50.307154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:50.307154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:50.307154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:50.307154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:50.307154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:50.307154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:50.307154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:50.307154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:50.307154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:50.307154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:50.308290 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:10:50.308290 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:10:50.308290 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:10:50.308290 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:10:50.308290 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:10:50.308290 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:10:50.308290 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:50.308290 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:50.311793 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 328/669] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:50.312621 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122b74f-5e6f-42a3-9134-017af6ac7488 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:50.328117 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7a9a2-7029-482e-8d0b-0676553fb4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:50.329298 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7a9a2-7029-482e-8d0b-0676553fb4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:50.329767 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7a9a2-7029-482e-8d0b-0676553fb4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:50.330108 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7a9a2-7029-482e-8d0b-0676553fb4d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:50.339787 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7a9a2-7029-482e-8d0b-0676553fb4d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:50.341247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7a9a2-7029-482e-8d0b-0676553fb4d1 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:50.352391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-55f7a9a2-7029-482e-8d0b-0676553fb4d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:50.352391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:50.352391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:50.352391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:50.352391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:50.352391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:50.352391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:50.352391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:50.352391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:50.352391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:50.352391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:50.352391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:50.352391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:10:50.352391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:10:50.352391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:10:50.353415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:10:50.353415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:50.353415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:50.353415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:50.353415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:50.353415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:50.353415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:50.353415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:50.353415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:50.353415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:50.353415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:50.353415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:50.353415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:50.353415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:50.353415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:50.353415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:50.353415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:50.354319 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:10:50.354319 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:10:50.354319 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:10:50.354319 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:10:50.354319 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:10:50.354319 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:10:50.354319 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:50.354319 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:50.354319 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 342/670] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:50.359415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495aa7f-aeed-491e-ba70-d67b4f3b767c None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:50.359775 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495aa7f-aeed-491e-ba70-d67b4f3b767c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:50.360053 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495aa7f-aeed-491e-ba70-d67b4f3b767c None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:50.361444 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 343/671] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:50.361995 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495aa7f-aeed-491e-ba70-d67b4f3b767c None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:50.401733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:50.428875 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122b74f-5e6f-42a3-9134-017af6ac7488 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:50.467049 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20327ea-f5fb-4e70-bb03-0e05ce8d3dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:50.467313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:50.467649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20327ea-f5fb-4e70-bb03-0e05ce8d3dac None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:50.467959 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20327ea-f5fb-4e70-bb03-0e05ce8d3dac None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:50.468237 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20327ea-f5fb-4e70-bb03-0e05ce8d3dac None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:50.468932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20327ea-f5fb-4e70-bb03-0e05ce8d3dac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:50.469336 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20327ea-f5fb-4e70-bb03-0e05ce8d3dac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:50.470921 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20327ea-f5fb-4e70-bb03-0e05ce8d3dac None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:50.480836 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495aa7f-aeed-491e-ba70-d67b4f3b767c None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:50.487555 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 329/672] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:10:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:50.487555 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20327ea-f5fb-4e70-bb03-0e05ce8d3dac None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:50.524051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec6ca4-bb17-406e-b587-34e530467e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:50.525303 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec6ca4-bb17-406e-b587-34e530467e4a None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:50.525664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec6ca4-bb17-406e-b587-34e530467e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:50.526001 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec6ca4-bb17-406e-b587-34e530467e4a None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:50.526762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec6ca4-bb17-406e-b587-34e530467e4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:50.527139 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec6ca4-bb17-406e-b587-34e530467e4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:50.528666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec6ca4-bb17-406e-b587-34e530467e4a None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:50.541150 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 344/673] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:10:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:50.542420 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27789ed0-a1ad-4bf6-9632-8abee1baa0bc None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:50.542713 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27789ed0-a1ad-4bf6-9632-8abee1baa0bc None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:50.542988 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27789ed0-a1ad-4bf6-9632-8abee1baa0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:50.561039 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-27789ed0-a1ad-4bf6-9632-8abee1baa0bc None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:50.562234 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-27789ed0-a1ad-4bf6-9632-8abee1baa0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:50.591864 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20327ea-f5fb-4e70-bb03-0e05ce8d3dac None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:50.624547 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef23d69-b57c-4658-992f-ea2f3f7b15df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:50.625378 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef23d69-b57c-4658-992f-ea2f3f7b15df None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:50.625656 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef23d69-b57c-4658-992f-ea2f3f7b15df None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:50.625926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef23d69-b57c-4658-992f-ea2f3f7b15df None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:50.626617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef23d69-b57c-4658-992f-ea2f3f7b15df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:50.626924 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef23d69-b57c-4658-992f-ea2f3f7b15df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:50.628402 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef23d69-b57c-4658-992f-ea2f3f7b15df None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:50.628833 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 345/674] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:50.629667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27789ed0-a1ad-4bf6-9632-8abee1baa0bc None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:50.630969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:50.647161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 330/675] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:10:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:50.648436 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217873b-8832-4c04-969c-0f55d66dc379 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:50.648887 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217873b-8832-4c04-969c-0f55d66dc379 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:50.649274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217873b-8832-4c04-969c-0f55d66dc379 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:50.652685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 331/676] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:50.654024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217873b-8832-4c04-969c-0f55d66dc379 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:50.715958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:50.738923 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27789ed0-a1ad-4bf6-9632-8abee1baa0bc None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:50.760280 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217873b-8832-4c04-969c-0f55d66dc379 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:50.776449 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52b3e6-ccea-44df-93dc-62ce3446d92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:50.777168 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52b3e6-ccea-44df-93dc-62ce3446d92e None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:50.777447 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52b3e6-ccea-44df-93dc-62ce3446d92e None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:50.777720 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52b3e6-ccea-44df-93dc-62ce3446d92e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:50.778439 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52b3e6-ccea-44df-93dc-62ce3446d92e None admin] RBAC: Adding query filter params (role.id=af9d39307d714527b14b2045ab466e7a, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=88670b0e1a0f460d8f74917c92db9b9a) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:50.778750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52b3e6-ccea-44df-93dc-62ce3446d92e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:50.780323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52b3e6-ccea-44df-93dc-62ce3446d92e None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:50.791795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 346/677] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:10:50 2022] GET /identity/v3/role_assignments?user.id=88670b0e1a0f460d8f74917c92db9b9a&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=af9d39307d714527b14b2045ab466e7a => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:50.791795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52b3e6-ccea-44df-93dc-62ce3446d92e None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:50.804637 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5215188-5f32-4925-a565-a6300ef85085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:50.805373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5215188-5f32-4925-a565-a6300ef85085 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:50.805717 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5215188-5f32-4925-a565-a6300ef85085 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:50.806142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5215188-5f32-4925-a565-a6300ef85085 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:50.806956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5215188-5f32-4925-a565-a6300ef85085 None admin] RBAC: Adding query filter params (role.id=af9d39307d714527b14b2045ab466e7a, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=f9f494df37254102b339df35a151e075) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:50.807326 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5215188-5f32-4925-a565-a6300ef85085 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:50.808940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5215188-5f32-4925-a565-a6300ef85085 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:50.816257 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 332/678] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:10:50 2022] GET /identity/v3/role_assignments?user.id=f9f494df37254102b339df35a151e075&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=af9d39307d714527b14b2045ab466e7a => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:50.817721 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356afb05-f279-4d99-a9f3-73934e27c289 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:50.818015 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356afb05-f279-4d99-a9f3-73934e27c289 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:50.818293 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356afb05-f279-4d99-a9f3-73934e27c289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:50.859088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-356afb05-f279-4d99-a9f3-73934e27c289 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:50.860447 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-356afb05-f279-4d99-a9f3-73934e27c289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:50.956983 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:50.958574 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52b3e6-ccea-44df-93dc-62ce3446d92e None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:50.972514 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 333/679] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:50.973775 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356afb05-f279-4d99-a9f3-73934e27c289 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:51.018020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadcc1e-0f91-4d2d-bd1f-7262244f6472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:51.018782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadcc1e-0f91-4d2d-bd1f-7262244f6472 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:51.019154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadcc1e-0f91-4d2d-bd1f-7262244f6472 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:51.019522 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadcc1e-0f91-4d2d-bd1f-7262244f6472 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:51.020404 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadcc1e-0f91-4d2d-bd1f-7262244f6472 None admin] RBAC: Adding query filter params (role.id=af9d39307d714527b14b2045ab466e7a, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=65de2e5f67dd4bb6a20cf3934afc8abd) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:51.020830 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadcc1e-0f91-4d2d-bd1f-7262244f6472 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:51.022383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadcc1e-0f91-4d2d-bd1f-7262244f6472 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:51.034441 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 347/680] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:10:50 2022] GET /identity/v3/role_assignments?user.id=65de2e5f67dd4bb6a20cf3934afc8abd&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=af9d39307d714527b14b2045ab466e7a => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:51.109140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356afb05-f279-4d99-a9f3-73934e27c289 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:51.149138 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1ff20-0d77-474e-8157-cde5c7183ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:51.150574 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1ff20-0d77-474e-8157-cde5c7183ccb None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:51.150999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1ff20-0d77-474e-8157-cde5c7183ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:51.151405 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1ff20-0d77-474e-8157-cde5c7183ccb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:51.163905 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1ff20-0d77-474e-8157-cde5c7183ccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:51.169148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-94c1ff20-0d77-474e-8157-cde5c7183ccb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:51.169148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:51.169148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:51.169148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:51.169148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:51.169148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:51.169148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:51.169148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:51.169148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:51.169148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:51.169148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:51.169148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:51.169148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:51.169148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:51.169148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:51.169148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:51.170306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:51.170306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:51.170306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:51.170306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:51.170306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:51.170306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:51.170306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:51.170306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:51.170306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:51.170306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:51.170306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:51.170306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:51.170306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:51.170306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:51.170306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:51.170306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:51.171325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:51.171325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:51.171325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:51.171325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:51.171325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:51.171325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:51.171325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:51.171325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:51.171325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:51.171325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:51.171325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:51.171325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:51.171325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 334/681] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:10:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:51.178068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadcc1e-0f91-4d2d-bd1f-7262244f6472 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:51.190178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:51.274008 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1ff20-0d77-474e-8157-cde5c7183ccb None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:51.312065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadcc1e-0f91-4d2d-bd1f-7262244f6472 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:51.332033 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:51.351655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3030bd6-b898-4218-914d-8d04aac9c5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:51.352410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3030bd6-b898-4218-914d-8d04aac9c5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:51.352685 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3030bd6-b898-4218-914d-8d04aac9c5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:51.352959 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3030bd6-b898-4218-914d-8d04aac9c5e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:51.353795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3030bd6-b898-4218-914d-8d04aac9c5e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:51.354103 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3030bd6-b898-4218-914d-8d04aac9c5e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:51.355625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3030bd6-b898-4218-914d-8d04aac9c5e7 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:51.365113 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 348/682] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:10:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:51.371729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3030bd6-b898-4218-914d-8d04aac9c5e7 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:51.393516 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1ff20-0d77-474e-8157-cde5c7183ccb None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:51.432734 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b9801-526a-474a-a10f-a0f6fc12cbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:51.433402 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b9801-526a-474a-a10f-a0f6fc12cbac None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:51.433634 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b9801-526a-474a-a10f-a0f6fc12cbac None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:51.433910 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b9801-526a-474a-a10f-a0f6fc12cbac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:51.444966 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b9801-526a-474a-a10f-a0f6fc12cbac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:51.450736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-f29b9801-526a-474a-a10f-a0f6fc12cbac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:10:51.450736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:51.450736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:51.450736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:51.450736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:51.450736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:51.450736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:51.450736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:51.450736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:51.450736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:51.450736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:51.450736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:51.450736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:51.450736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:51.450736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:51.450736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:51.451843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:51.451843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:51.451843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:51.451843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:51.451843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:51.451843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:51.451843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:51.451843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:51.451843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:51.451843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:51.451843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:51.451843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:51.451843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:51.451843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:51.451843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:51.451843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:51.452845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:51.452845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:51.452845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:51.452845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:51.452845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:51.452845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:51.452845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:51.452845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:51.452845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:51.452845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:51.452845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:10:51.452845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:51.453894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 335/683] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:10:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:51.459316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b9801-526a-474a-a10f-a0f6fc12cbac None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:51.460165 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:51.502066 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3030bd6-b898-4218-914d-8d04aac9c5e7 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:51.548057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecfbed-9a3b-40f5-83de-c0f244f5bde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:51.548612 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecfbed-9a3b-40f5-83de-c0f244f5bde3 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:51.548889 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecfbed-9a3b-40f5-83de-c0f244f5bde3 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:51.549166 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecfbed-9a3b-40f5-83de-c0f244f5bde3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:51.563497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feecfbed-9a3b-40f5-83de-c0f244f5bde3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:51.564983 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feecfbed-9a3b-40f5-83de-c0f244f5bde3 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:51.581500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-feecfbed-9a3b-40f5-83de-c0f244f5bde3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:51.581500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:51.581500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:51.581500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:51.581500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:51.581500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:51.581500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:51.581500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:51.581500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:51.581500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:51.581500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:51.581500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:51.581500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:51.581500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:51.581500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:51.581500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:51.582494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:51.582494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:51.582494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:51.582494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:51.582494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:51.582494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:51.582494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:51.582494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:51.582494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:51.582494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:51.582494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:51.582494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:51.582494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:51.582494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:51.582494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:51.582494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:51.582494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:51.583437 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:51.583437 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:51.583437 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:51.587616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 349/684] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:10:51.589207 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecfbed-9a3b-40f5-83de-c0f244f5bde3 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:51.599003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b9801-526a-474a-a10f-a0f6fc12cbac None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:51.612485 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:51.639049 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bddba-0671-4b9f-a170-3084cdcd3a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:51.639658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bddba-0671-4b9f-a170-3084cdcd3a58 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:51.639993 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bddba-0671-4b9f-a170-3084cdcd3a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:51.640274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bddba-0671-4b9f-a170-3084cdcd3a58 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:51.640944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5bddba-0671-4b9f-a170-3084cdcd3a58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:51.641252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5bddba-0671-4b9f-a170-3084cdcd3a58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:51.642637 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5bddba-0671-4b9f-a170-3084cdcd3a58 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:51.655882 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 336/685] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:51.662880 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bddba-0671-4b9f-a170-3084cdcd3a58 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:51.736490 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecfbed-9a3b-40f5-83de-c0f244f5bde3 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:51.774017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c851e13-b64e-49b2-994e-a1a167cd4b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:51.774687 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c851e13-b64e-49b2-994e-a1a167cd4b38 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:51.774971 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c851e13-b64e-49b2-994e-a1a167cd4b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:51.775246 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c851e13-b64e-49b2-994e-a1a167cd4b38 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:51.775946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c851e13-b64e-49b2-994e-a1a167cd4b38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:51.776256 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c851e13-b64e-49b2-994e-a1a167cd4b38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:51.777933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c851e13-b64e-49b2-994e-a1a167cd4b38 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:51.788125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:51.796254 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 350/686] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:51.799155 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c851e13-b64e-49b2-994e-a1a167cd4b38 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:51.909292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c851e13-b64e-49b2-994e-a1a167cd4b38 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:51.929446 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bddba-0671-4b9f-a170-3084cdcd3a58 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:51.948740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab97bbd-04dc-4b0e-a970-069559d8fa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:51.949555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab97bbd-04dc-4b0e-a970-069559d8fa9a None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:51.949909 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab97bbd-04dc-4b0e-a970-069559d8fa9a None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:51.950255 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab97bbd-04dc-4b0e-a970-069559d8fa9a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:51.958975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab97bbd-04dc-4b0e-a970-069559d8fa9a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:51.960511 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab97bbd-04dc-4b0e-a970-069559d8fa9a None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:51.984269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-9ab97bbd-04dc-4b0e-a970-069559d8fa9a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:10:51.984269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:51.984269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:51.984269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:51.984269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:51.984269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:51.984269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:51.984269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:51.984269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:51.984269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:51.984269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:51.984269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:51.984269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:10:51.984269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:10:51.984269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:10:51.984269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:10:51.985304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:51.985304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:51.985304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:10:51.985304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:51.985304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:10:51.985304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:51.985304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:51.985304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:51.985304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:51.985304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:51.985304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:51.985304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:51.985304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:51.985304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:51.985304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:51.985304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:51.985304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:51.985304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:51.986287 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:51.986287 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:51.986287 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:10:51.986287 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:10:51.986287 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:10:51.986287 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:10:51.986287 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:10:51.986287 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:10:51.986287 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:10:51.986287 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:51.986287 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 351/687] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:10:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:10:51.991490 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab97bbd-04dc-4b0e-a970-069559d8fa9a None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:52.001979 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab8244-99d3-4d0c-8a65-455c2e598b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:52.002623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab8244-99d3-4d0c-8a65-455c2e598b25 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:52.002905 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab8244-99d3-4d0c-8a65-455c2e598b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:52.003231 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab8244-99d3-4d0c-8a65-455c2e598b25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:52.022530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab8244-99d3-4d0c-8a65-455c2e598b25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:52.024904 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab8244-99d3-4d0c-8a65-455c2e598b25 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:52.042558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:52.056337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-6fab8244-99d3-4d0c-8a65-455c2e598b25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:52.056337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:52.056337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:52.056337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:52.056337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:52.056337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:52.056337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:52.056337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:52.056337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:52.056337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:52.056337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:52.056337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:52.056337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:52.056337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:52.056337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:52.056337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:52.057299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:52.057299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:52.057299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:52.057299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:52.057299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:52.057299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:52.057299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:52.057299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:52.057299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:52.057299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:52.057299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:52.057299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:52.057299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:52.057299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:52.057299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:52.057299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:52.057299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:52.058219 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:52.058219 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:52.058219 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:52.066075 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 337/688] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:52.067633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab8244-99d3-4d0c-8a65-455c2e598b25 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:52.121631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:52.165218 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab97bbd-04dc-4b0e-a970-069559d8fa9a None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:52.204214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19d25e-ee73-47dd-b889-97bbc70f7617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:52.204880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19d25e-ee73-47dd-b889-97bbc70f7617 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:52.205138 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19d25e-ee73-47dd-b889-97bbc70f7617 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:52.205410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19d25e-ee73-47dd-b889-97bbc70f7617 None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:52.206136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19d25e-ee73-47dd-b889-97bbc70f7617 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:52.206440 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19d25e-ee73-47dd-b889-97bbc70f7617 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:52.207878 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19d25e-ee73-47dd-b889-97bbc70f7617 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:52.228517 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 352/689] 10.222.0.81 () {56 vars in 1135 bytes} [Mon Sep 19 11:10:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:52.234516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19d25e-ee73-47dd-b889-97bbc70f7617 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:52.297700 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab8244-99d3-4d0c-8a65-455c2e598b25 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:52.367094 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a187e-849f-44a9-bbbb-0a92c627644d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:52.367709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a187e-849f-44a9-bbbb-0a92c627644d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:52.368112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a187e-849f-44a9-bbbb-0a92c627644d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:52.368387 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a187e-849f-44a9-bbbb-0a92c627644d None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:52.369040 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a187e-849f-44a9-bbbb-0a92c627644d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:52.369346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a187e-849f-44a9-bbbb-0a92c627644d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:52.370726 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a187e-849f-44a9-bbbb-0a92c627644d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:52.397055 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 338/690] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:52.400299 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:52.402591 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a187e-849f-44a9-bbbb-0a92c627644d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:52.408088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19d25e-ee73-47dd-b889-97bbc70f7617 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:52.480866 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09e0ab-9ca5-4c1a-80ec-f9200979f5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:52.481762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09e0ab-9ca5-4c1a-80ec-f9200979f5c8 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:52.482244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09e0ab-9ca5-4c1a-80ec-f9200979f5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:52.482687 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09e0ab-9ca5-4c1a-80ec-f9200979f5c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:52.533383 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:52.563359 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09e0ab-9ca5-4c1a-80ec-f9200979f5c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:52.565391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09e0ab-9ca5-4c1a-80ec-f9200979f5c8 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:52.565908 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a187e-849f-44a9-bbbb-0a92c627644d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:52.630221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6f3ec-0c1b-4f57-801f-8e123f62a113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:52.630961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6f3ec-0c1b-4f57-801f-8e123f62a113 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:52.631314 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6f3ec-0c1b-4f57-801f-8e123f62a113 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:52.631665 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6f3ec-0c1b-4f57-801f-8e123f62a113 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:52.646363 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6f3ec-0c1b-4f57-801f-8e123f62a113 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:52.648324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6f3ec-0c1b-4f57-801f-8e123f62a113 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:52.677417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-b5f6f3ec-0c1b-4f57-801f-8e123f62a113 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:10:52.677417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:52.677417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:52.677417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:52.677417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:52.677417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:52.677417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:52.677417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:52.677417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:52.677417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:52.677417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:52.677417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:52.677417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:10:52.677417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:10:52.677417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:10:52.677417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:10:52.678617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:52.678617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:52.678617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:10:52.678617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:52.678617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:10:52.678617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:52.678617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:52.678617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:52.678617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:52.678617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:52.678617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:52.678617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:52.678617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:52.678617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:52.678617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:52.678617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:52.678617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:52.678617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:52.679753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:52.679753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:52.679753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:10:52.679753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:10:52.679753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:10:52.679753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:10:52.679753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:10:52.679753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:10:52.679753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:10:52.679753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:52.689044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 339/691] 10.222.0.81 () {56 vars in 1093 bytes} [Mon Sep 19 11:10:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:52.693147 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6f3ec-0c1b-4f57-801f-8e123f62a113 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:52.704179 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:52.708692 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 353/692] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 11:10:52.720790 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09e0ab-9ca5-4c1a-80ec-f9200979f5c8 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:52.817072 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6f3ec-0c1b-4f57-801f-8e123f62a113 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:52.854501 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ab7ff-4ab8-49e1-be89-38d1356bd428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:52.855253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ab7ff-4ab8-49e1-be89-38d1356bd428 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:52.855633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ab7ff-4ab8-49e1-be89-38d1356bd428 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:52.856150 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ab7ff-4ab8-49e1-be89-38d1356bd428 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:52.857027 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ab7ff-4ab8-49e1-be89-38d1356bd428 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:52.857438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ab7ff-4ab8-49e1-be89-38d1356bd428 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:52.859040 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ab7ff-4ab8-49e1-be89-38d1356bd428 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:52.878834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 340/693] 10.222.0.81 () {56 vars in 1133 bytes} [Mon Sep 19 11:10:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:52.884138 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ab7ff-4ab8-49e1-be89-38d1356bd428 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:52.887060 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:52.937957 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09e0ab-9ca5-4c1a-80ec-f9200979f5c8 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:52.992027 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ab7ff-4ab8-49e1-be89-38d1356bd428 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:53.028176 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2713b7a-ee42-4e5f-8a6e-ba730e44c8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:53.029277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2713b7a-ee42-4e5f-8a6e-ba730e44c8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:53.029766 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2713b7a-ee42-4e5f-8a6e-ba730e44c8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:53.030245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2713b7a-ee42-4e5f-8a6e-ba730e44c8d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:53.084496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac58d9-79f7-4dd2-9324-be2c72d16fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:53.085146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac58d9-79f7-4dd2-9324-be2c72d16fdb None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:53.085426 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac58d9-79f7-4dd2-9324-be2c72d16fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:53.085701 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac58d9-79f7-4dd2-9324-be2c72d16fdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:53.104435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2713b7a-ee42-4e5f-8a6e-ba730e44c8d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:53.106275 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2713b7a-ee42-4e5f-8a6e-ba730e44c8d2 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:53.124744 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac58d9-79f7-4dd2-9324-be2c72d16fdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:53.126326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac58d9-79f7-4dd2-9324-be2c72d16fdb None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:53.132173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:53.169567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-63ac58d9-79f7-4dd2-9324-be2c72d16fdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:53.169567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:53.169567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:53.169567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:53.169567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:53.169567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:53.169567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:53.169567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:53.169567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:53.169567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:53.169567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:53.169567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:53.169567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:53.169567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:53.169567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:53.169567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:53.170559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:53.170559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:53.170559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:53.170559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:53.170559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:53.170559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:53.170559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:53.170559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:53.170559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:53.170559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:53.170559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:53.170559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:53.170559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:53.170559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:53.170559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:53.170559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:53.170559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:53.171457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:53.171457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:53.171457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:53.180228 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 354/694] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 460 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:53.186288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac58d9-79f7-4dd2-9324-be2c72d16fdb None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:53.236987 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 341/695] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:10:53.239928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2713b7a-ee42-4e5f-8a6e-ba730e44c8d2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:53.299736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:53.371842 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac58d9-79f7-4dd2-9324-be2c72d16fdb None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:53.382333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2713b7a-ee42-4e5f-8a6e-ba730e44c8d2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:53.404197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:53.416982 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba5a82-a5f6-4e2f-bb05-dbd0ee2e2d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:53.417893 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba5a82-a5f6-4e2f-bb05-dbd0ee2e2d96 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:53.418185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba5a82-a5f6-4e2f-bb05-dbd0ee2e2d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:53.418696 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba5a82-a5f6-4e2f-bb05-dbd0ee2e2d96 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:53.419342 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba5a82-a5f6-4e2f-bb05-dbd0ee2e2d96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:53.419696 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba5a82-a5f6-4e2f-bb05-dbd0ee2e2d96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:53.421358 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba5a82-a5f6-4e2f-bb05-dbd0ee2e2d96 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:53.426339 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f2a05-cb38-422d-a99d-b9ec3e805ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:53.427224 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f2a05-cb38-422d-a99d-b9ec3e805ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:53.427844 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f2a05-cb38-422d-a99d-b9ec3e805ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:53.428338 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f2a05-cb38-422d-a99d-b9ec3e805ed2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:53.434052 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 355/696] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:53.442390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba5a82-a5f6-4e2f-bb05-dbd0ee2e2d96 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:53.446272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f2a05-cb38-422d-a99d-b9ec3e805ed2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:53.447736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f2a05-cb38-422d-a99d-b9ec3e805ed2 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:53.466065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-026f2a05-cb38-422d-a99d-b9ec3e805ed2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:53.466065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:53.466065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:53.466065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:53.466065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:53.466065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:53.466065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:53.466065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:53.466065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:53.466065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:53.466065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:53.466065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:53.466065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:53.466065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:53.466065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:53.466065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:53.467078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:53.467078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:53.467078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:53.467078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:53.467078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:53.467078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:53.467078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:53.467078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:53.467078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:53.467078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:53.467078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:53.467078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:53.467078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:53.467078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:53.467078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:53.467078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:53.467078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:53.468614 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:53.468614 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:53.468614 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:53.468614 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 342/697] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:53.472823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f2a05-cb38-422d-a99d-b9ec3e805ed2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:53.577279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba5a82-a5f6-4e2f-bb05-dbd0ee2e2d96 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:53.591934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:53.615121 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd23b95-8fc4-4f28-b46d-99e6c751a47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:53.615776 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd23b95-8fc4-4f28-b46d-99e6c751a47c None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:53.616244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd23b95-8fc4-4f28-b46d-99e6c751a47c None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:53.616354 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd23b95-8fc4-4f28-b46d-99e6c751a47c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:53.623656 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd23b95-8fc4-4f28-b46d-99e6c751a47c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:53.624025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f2a05-cb38-422d-a99d-b9ec3e805ed2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:53.625191 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd23b95-8fc4-4f28-b46d-99e6c751a47c None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:53.636649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-fcd23b95-8fc4-4f28-b46d-99e6c751a47c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:53.636649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:53.636649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:53.636649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:53.636649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:53.636649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:53.636649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:53.636649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:53.636649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:53.636649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:53.636649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:53.636649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:53.636649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:10:53.636649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:10:53.636649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:10:53.637569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:10:53.637569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:53.637569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:53.637569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:53.637569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:53.637569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:53.637569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:53.637569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:53.637569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:53.637569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:53.637569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:53.637569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:53.637569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:53.637569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:53.637569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:53.637569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:53.637569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:53.638450 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:10:53.638450 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:10:53.638450 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:10:53.638450 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:10:53.638450 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:10:53.638450 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:10:53.638450 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:53.638450 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:53.642490 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 356/698] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:10:53.643581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1b257-6e5c-4c2f-97ba-85de1b292f71 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:53.643888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1b257-6e5c-4c2f-97ba-85de1b292f71 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:53.644162 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1b257-6e5c-4c2f-97ba-85de1b292f71 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:53.645512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 357/699] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:53.646410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8255c3c-34be-4cdb-8556-6e415a17be77 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:53.646686 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8255c3c-34be-4cdb-8556-6e415a17be77 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:53.646952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8255c3c-34be-4cdb-8556-6e415a17be77 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:53.648319 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 358/700] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:53.650050 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8255c3c-34be-4cdb-8556-6e415a17be77 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:53.668673 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5ad77-c224-47bc-820d-6fbe0b8494b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:53.669414 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5ad77-c224-47bc-820d-6fbe0b8494b7 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:53.669594 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5ad77-c224-47bc-820d-6fbe0b8494b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:53.669910 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5ad77-c224-47bc-820d-6fbe0b8494b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:53.670494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5ad77-c224-47bc-820d-6fbe0b8494b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:53.670796 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5ad77-c224-47bc-820d-6fbe0b8494b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:53.672554 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5ad77-c224-47bc-820d-6fbe0b8494b7 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:53.684389 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 343/701] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:53.685671 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28537b1f-3583-481a-89c1-2a43705b2e84 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:53.685963 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28537b1f-3583-481a-89c1-2a43705b2e84 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:53.686238 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28537b1f-3583-481a-89c1-2a43705b2e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:53.705644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-28537b1f-3583-481a-89c1-2a43705b2e84 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:53.706830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-28537b1f-3583-481a-89c1-2a43705b2e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:53.716463 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:53.765530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8255c3c-34be-4cdb-8556-6e415a17be77 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:53.778744 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 344/702] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:53.780207 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62d968-99f4-478c-a003-68cee85a10ba None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:53.780685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62d968-99f4-478c-a003-68cee85a10ba None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:53.781142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62d968-99f4-478c-a003-68cee85a10ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:53.799795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6a62d968-99f4-478c-a003-68cee85a10ba None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:53.801294 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6a62d968-99f4-478c-a003-68cee85a10ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:53.805194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac6452-33b6-4112-920b-98b84658e67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:53.806348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac6452-33b6-4112-920b-98b84658e67f None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:53.806628 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac6452-33b6-4112-920b-98b84658e67f None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:53.806948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac6452-33b6-4112-920b-98b84658e67f None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:53.807639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac6452-33b6-4112-920b-98b84658e67f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:53.807965 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac6452-33b6-4112-920b-98b84658e67f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:53.809371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac6452-33b6-4112-920b-98b84658e67f None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:53.830417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 359/703] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:10:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:53.830417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac6452-33b6-4112-920b-98b84658e67f None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:53.865932 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:53.894700 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 345/704] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:53.895991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62d968-99f4-478c-a003-68cee85a10ba None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:53.958711 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac6452-33b6-4112-920b-98b84658e67f None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:53.990538 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:54.004373 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb245d-3485-49a3-9735-16ca82c24f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:54.005333 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb245d-3485-49a3-9735-16ca82c24f31 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:54.005851 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb245d-3485-49a3-9735-16ca82c24f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:54.006338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb245d-3485-49a3-9735-16ca82c24f31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:54.014243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb245d-3485-49a3-9735-16ca82c24f31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:54.015900 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb245d-3485-49a3-9735-16ca82c24f31 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:54.023314 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62d968-99f4-478c-a003-68cee85a10ba None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:54.028429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-93eb245d-3485-49a3-9735-16ca82c24f31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:54.028429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:54.028429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:54.028429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:54.028429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:54.028429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:54.028429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:54.028429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:54.028429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:54.028429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:54.028429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:54.028429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:54.028429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:10:54.028429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:10:54.028429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:10:54.029340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:10:54.029340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:54.029340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:54.029340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:54.029340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:54.029340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:54.029340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:54.029340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:54.029340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:54.029340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:54.029340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:54.029340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:54.029340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:54.029340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:54.029340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:54.029340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:54.029340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:54.030212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:10:54.030212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:10:54.030212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:10:54.030212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:10:54.030212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:10:54.030212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:10:54.030212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:10:54.030212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:54.032569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 360/705] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:54.033925 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5ead9-4dc2-416d-9cfb-a8b71da0daee None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:54.034418 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5ead9-4dc2-416d-9cfb-a8b71da0daee None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:54.034882 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5ead9-4dc2-416d-9cfb-a8b71da0daee None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:54.036591 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 361/706] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:54.042964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80d1b4-37c3-49de-b7cc-3a98e523a2e8 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:54.043254 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80d1b4-37c3-49de-b7cc-3a98e523a2e8 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:54.043536 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80d1b4-37c3-49de-b7cc-3a98e523a2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:54.064698 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-8a80d1b4-37c3-49de-b7cc-3a98e523a2e8 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:54.065363 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0aacdb-0aeb-4bcf-a38e-aa4d95e0185f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:54.065889 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-8a80d1b4-37c3-49de-b7cc-3a98e523a2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:54.066394 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0aacdb-0aeb-4bcf-a38e-aa4d95e0185f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:54.066887 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0aacdb-0aeb-4bcf-a38e-aa4d95e0185f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:54.067374 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0aacdb-0aeb-4bcf-a38e-aa4d95e0185f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:54.068350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0aacdb-0aeb-4bcf-a38e-aa4d95e0185f None admin] RBAC: Adding query filter params (role.id=af9d39307d714527b14b2045ab466e7a, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=1c8a9732807d4921a41a6a2d5019e2f5) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:54.068899 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0aacdb-0aeb-4bcf-a38e-aa4d95e0185f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:54.070502 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0aacdb-0aeb-4bcf-a38e-aa4d95e0185f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:54.079890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 346/707] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:10:53 2022] GET /identity/v3/role_assignments?user.id=1c8a9732807d4921a41a6a2d5019e2f5&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=af9d39307d714527b14b2045ab466e7a => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:54.080820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0aacdb-0aeb-4bcf-a38e-aa4d95e0185f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:54.160035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 362/708] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 11:10:54.161018 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80d1b4-37c3-49de-b7cc-3a98e523a2e8 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:54.186334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:54.215986 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0aacdb-0aeb-4bcf-a38e-aa4d95e0185f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:54.264637 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84076f-d70b-4546-9ff0-081e16a07ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:54.265935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84076f-d70b-4546-9ff0-081e16a07ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:54.267004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84076f-d70b-4546-9ff0-081e16a07ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:54.267490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84076f-d70b-4546-9ff0-081e16a07ee1 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:54.268846 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84076f-d70b-4546-9ff0-081e16a07ee1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:54.269772 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84076f-d70b-4546-9ff0-081e16a07ee1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:54.272762 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84076f-d70b-4546-9ff0-081e16a07ee1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:54.277357 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80d1b4-37c3-49de-b7cc-3a98e523a2e8 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:54.284114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:54.293206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 347/709] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:10:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:54.295135 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84076f-d70b-4546-9ff0-081e16a07ee1 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:54.319021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9364bb2-9336-4c82-9737-2b4a6643c412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:54.319664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9364bb2-9336-4c82-9737-2b4a6643c412 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:54.319980 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9364bb2-9336-4c82-9737-2b4a6643c412 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:54.320339 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9364bb2-9336-4c82-9737-2b4a6643c412 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:54.332465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9364bb2-9336-4c82-9737-2b4a6643c412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:54.333708 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9364bb2-9336-4c82-9737-2b4a6643c412 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:54.353921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-e9364bb2-9336-4c82-9737-2b4a6643c412 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:54.353921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:54.353921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:54.353921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:54.353921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:54.353921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:54.353921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:54.353921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:54.353921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:54.353921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:54.353921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:54.353921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:54.353921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:54.353921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:54.353921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:54.353921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:54.354945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:54.354945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:54.354945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:54.354945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:54.354945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:54.354945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:54.354945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:54.354945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:54.354945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:54.354945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:54.354945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:54.354945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:54.354945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:54.354945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:54.354945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:54.354945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:54.354945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:54.355926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:54.355926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:54.355926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:54.356154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 363/710] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:54.357794 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9364bb2-9336-4c82-9737-2b4a6643c412 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:54.431105 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84076f-d70b-4546-9ff0-081e16a07ee1 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:54.451659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:54.470688 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124de93-30ca-4351-b93a-57affcf4e2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:54.471283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124de93-30ca-4351-b93a-57affcf4e2bd None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:54.471516 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124de93-30ca-4351-b93a-57affcf4e2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:54.471830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124de93-30ca-4351-b93a-57affcf4e2bd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:54.472530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e124de93-30ca-4351-b93a-57affcf4e2bd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:54.474079 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e124de93-30ca-4351-b93a-57affcf4e2bd None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:54.479541 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9364bb2-9336-4c82-9737-2b4a6643c412 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:54.485578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-e124de93-30ca-4351-b93a-57affcf4e2bd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:10:54.485578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:54.485578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:10:54.485578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:10:54.485578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:10:54.485578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:10:54.485578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:10:54.485578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:10:54.485578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:10:54.485578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:54.485578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:10:54.485578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:54.485578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:54.485578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:54.485578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:54.485578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:54.485578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:54.486603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:54.486603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:54.486603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:54.486603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:54.486603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:54.486603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:54.486603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:10:54.486603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:10:54.486603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:10:54.486603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:10:54.486603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:54.486603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:54.486603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:54.486603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:54.486603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:54.486603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:54.487503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:54.487503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:54.487503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:54.487503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:54.487503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:54.487503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:54.487503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:54.487503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:54.487503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:54.487503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:54.487503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:10:54.487503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:10:54.487503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:10:54.487503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:54.487503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 348/711] 10.222.0.81 () {56 vars in 1129 bytes} [Mon Sep 19 11:10:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:54.488891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124de93-30ca-4351-b93a-57affcf4e2bd None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:54.518979 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aaf2a0-b839-47d0-887c-ea2ca9dfaa32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:54.519585 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aaf2a0-b839-47d0-887c-ea2ca9dfaa32 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:54.519892 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aaf2a0-b839-47d0-887c-ea2ca9dfaa32 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:54.520164 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aaf2a0-b839-47d0-887c-ea2ca9dfaa32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:54.520889 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aaf2a0-b839-47d0-887c-ea2ca9dfaa32 None admin] RBAC: Adding query filter params (role.id=753f8b35d5f0458d8f62d0ad6e28033f, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=5abd70a18ccd407bafe3b61c8870986e) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:54.521193 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aaf2a0-b839-47d0-887c-ea2ca9dfaa32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:54.522608 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aaf2a0-b839-47d0-887c-ea2ca9dfaa32 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:54.535805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 364/712] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:10:54 2022] GET /identity/v3/role_assignments?user.id=5abd70a18ccd407bafe3b61c8870986e&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=753f8b35d5f0458d8f62d0ad6e28033f => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:54.535805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aaf2a0-b839-47d0-887c-ea2ca9dfaa32 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:54.577696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:54.641173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124de93-30ca-4351-b93a-57affcf4e2bd None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:54.665467 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aaf2a0-b839-47d0-887c-ea2ca9dfaa32 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:54.687202 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33d9a1-1611-43f4-b75d-2d79f1ef81de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:54.688126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33d9a1-1611-43f4-b75d-2d79f1ef81de None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:54.688406 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33d9a1-1611-43f4-b75d-2d79f1ef81de None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:54.688729 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33d9a1-1611-43f4-b75d-2d79f1ef81de None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:54.689550 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33d9a1-1611-43f4-b75d-2d79f1ef81de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:54.690177 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33d9a1-1611-43f4-b75d-2d79f1ef81de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:54.691603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33d9a1-1611-43f4-b75d-2d79f1ef81de None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:54.709762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177757c-ba57-4e3b-8518-e6d8fb4e5259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:54.710555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177757c-ba57-4e3b-8518-e6d8fb4e5259 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:54.710913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177757c-ba57-4e3b-8518-e6d8fb4e5259 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:54.711279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177757c-ba57-4e3b-8518-e6d8fb4e5259 None admin] PATH_INFO: `/v3/services/network` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:54.712028 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177757c-ba57-4e3b-8518-e6d8fb4e5259 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:54.713486 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177757c-ba57-4e3b-8518-e6d8fb4e5259 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:54.714235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 349/713] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:54.715318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33d9a1-1611-43f4-b75d-2d79f1ef81de None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:54.729383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-1177757c-ba57-4e3b-8518-e6d8fb4e5259 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:10:54.729383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:54.729383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:10:54.729383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:10:54.729383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:10:54.729383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:10:54.729383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:10:54.729383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:10:54.729383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:10:54.729383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:54.729383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:10:54.729383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:54.729383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:54.729383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:54.729383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:54.729383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:54.729383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:54.730638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:54.730638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:54.730638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:54.730638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:54.730638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:54.730638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:54.730638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:10:54.730638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:10:54.730638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:10:54.730638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:10:54.730638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:54.730638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:54.730638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:54.730638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:54.730638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:54.730638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:54.731746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:54.731746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:54.731746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:54.731746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:54.731746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:54.731746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:54.731746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:54.731746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:54.731746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:54.731746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:54.731746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:10:54.731746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:10:54.731746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:10:54.731746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:54.731746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 365/714] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:54 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:54.732924 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177757c-ba57-4e3b-8518-e6d8fb4e5259 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:54.738934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:54.848446 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33d9a1-1611-43f4-b75d-2d79f1ef81de None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:54.862201 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177757c-ba57-4e3b-8518-e6d8fb4e5259 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:54.872561 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:54.886067 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a5d4e-e45a-4f7c-b62d-9474ac7062d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:54.886675 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a5d4e-e45a-4f7c-b62d-9474ac7062d2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:54.886951 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a5d4e-e45a-4f7c-b62d-9474ac7062d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:54.887240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a5d4e-e45a-4f7c-b62d-9474ac7062d2 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:54.887876 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a5d4e-e45a-4f7c-b62d-9474ac7062d2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:54.888177 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a5d4e-e45a-4f7c-b62d-9474ac7062d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:54.889631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a5d4e-e45a-4f7c-b62d-9474ac7062d2 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:54.899058 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 350/715] 10.222.0.81 () {56 vars in 1124 bytes} [Mon Sep 19 11:10:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:54.899990 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a5d4e-e45a-4f7c-b62d-9474ac7062d2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:54.910096 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270e86c-9e43-42fe-8349-5eee26611adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:54.910726 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270e86c-9e43-42fe-8349-5eee26611adf None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:54.911060 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270e86c-9e43-42fe-8349-5eee26611adf None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:54.911329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270e86c-9e43-42fe-8349-5eee26611adf None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:54.912099 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270e86c-9e43-42fe-8349-5eee26611adf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:54.913528 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270e86c-9e43-42fe-8349-5eee26611adf None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:54.944146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 366/716] 10.222.0.81 () {60 vars in 1132 bytes} [Mon Sep 19 11:10:54 2022] POST /identity/v3/users => generated 254 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:10:54.945149 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270e86c-9e43-42fe-8349-5eee26611adf None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:55.055131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:55.079095 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a5d4e-e45a-4f7c-b62d-9474ac7062d2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:55.139461 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270e86c-9e43-42fe-8349-5eee26611adf None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:55.159037 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f2853-58c8-446d-8de6-67ba1b879d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:55.159622 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f2853-58c8-446d-8de6-67ba1b879d37 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:55.159926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f2853-58c8-446d-8de6-67ba1b879d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:55.160199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f2853-58c8-446d-8de6-67ba1b879d37 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:55.160866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f2853-58c8-446d-8de6-67ba1b879d37 None admin] RBAC: Adding query filter params (name=network) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:55.161182 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f2853-58c8-446d-8de6-67ba1b879d37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:55.162742 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f2853-58c8-446d-8de6-67ba1b879d37 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:55.176236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 351/717] 10.222.0.81 () {56 vars in 1112 bytes} [Mon Sep 19 11:10:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:55.181712 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f2853-58c8-446d-8de6-67ba1b879d37 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:55.197061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c012cc-3782-445c-8118-ab5b9459fcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:55.198245 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c012cc-3782-445c-8118-ab5b9459fcb0 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:55.199219 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c012cc-3782-445c-8118-ab5b9459fcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:55.199704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c012cc-3782-445c-8118-ab5b9459fcb0 None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:55.200672 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c012cc-3782-445c-8118-ab5b9459fcb0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:55.201183 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c012cc-3782-445c-8118-ab5b9459fcb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:55.202903 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c012cc-3782-445c-8118-ab5b9459fcb0 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:55.210127 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 367/718] 10.222.0.81 () {56 vars in 1124 bytes} [Mon Sep 19 11:10:54 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:55.248806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:55.310227 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f2853-58c8-446d-8de6-67ba1b879d37 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:55.350558 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff082e-f72d-425a-b881-c6e47efc169e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:55.351541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff082e-f72d-425a-b881-c6e47efc169e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:55.352059 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff082e-f72d-425a-b881-c6e47efc169e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:55.352568 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff082e-f72d-425a-b881-c6e47efc169e None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:55.353439 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff082e-f72d-425a-b881-c6e47efc169e None admin] RBAC: Adding query filter params (type=network) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:55.353970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff082e-f72d-425a-b881-c6e47efc169e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:55.355587 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff082e-f72d-425a-b881-c6e47efc169e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:55.362616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 352/719] 10.222.0.81 () {56 vars in 1112 bytes} [Mon Sep 19 11:10:55 2022] GET /identity/v3/services?type=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:10:56.687841 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21d650-e0e6-4929-b09d-58e81acf3007 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:56.688833 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21d650-e0e6-4929-b09d-58e81acf3007 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:56.689328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21d650-e0e6-4929-b09d-58e81acf3007 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:56.691130 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 368/720] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:56.696976 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4700cb-d0e4-4a83-8814-264a909c39b8 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:56.697820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4700cb-d0e4-4a83-8814-264a909c39b8 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:56.698324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4700cb-d0e4-4a83-8814-264a909c39b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:56.719631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-da4700cb-d0e4-4a83-8814-264a909c39b8 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:56.721166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-da4700cb-d0e4-4a83-8814-264a909c39b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:56.817876 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 353/721] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:57.131928 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee2771-91ce-4165-aa7a-49282e99e9ac None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:57.132610 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee2771-91ce-4165-aa7a-49282e99e9ac None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:57.133156 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee2771-91ce-4165-aa7a-49282e99e9ac None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:57.134952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 369/722] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:57.140416 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e98f70-2370-44dc-b55f-ce1080bf3ce9 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:57.140922 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e98f70-2370-44dc-b55f-ce1080bf3ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:57.141442 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e98f70-2370-44dc-b55f-ce1080bf3ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:57.147455 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee2771-91ce-4165-aa7a-49282e99e9ac None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:57.168952 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f0e98f70-2370-44dc-b55f-ce1080bf3ce9 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:57.170151 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f0e98f70-2370-44dc-b55f-ce1080bf3ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:57.248104 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:57.312822 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 354/723] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:57.313956 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee2771-91ce-4165-aa7a-49282e99e9ac None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:57.340099 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:57.355336 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2f7be-67ba-4de4-8ee0-dc4b0c7f6e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:57.356001 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2f7be-67ba-4de4-8ee0-dc4b0c7f6e08 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:57.356282 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2f7be-67ba-4de4-8ee0-dc4b0c7f6e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:57.356554 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2f7be-67ba-4de4-8ee0-dc4b0c7f6e08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:57.373658 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b2f7be-67ba-4de4-8ee0-dc4b0c7f6e08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:57.378477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-20b2f7be-67ba-4de4-8ee0-dc4b0c7f6e08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:10:57.378477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:57.378477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:57.378477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:57.378477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:57.378477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:57.378477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:57.378477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:57.378477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:57.378477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:57.378477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:57.378477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:57.378477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:57.378477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:57.378477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:57.378477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:57.379411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:57.379411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:57.379411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:57.379411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:57.379411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:57.379411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:57.379411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:57.379411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:57.379411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:57.379411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:57.379411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:57.379411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:57.379411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:57.379411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:57.379411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:57.379411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:57.380341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:57.380341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:57.380341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:57.380341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:57.380341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:57.380341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:57.380341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:57.380341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:57.380341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:57.380341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:57.380341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:10:57.380341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:57.380341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 370/724] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:10:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:57.385665 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e98f70-2370-44dc-b55f-ce1080bf3ce9 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:57.514046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e98f70-2370-44dc-b55f-ce1080bf3ce9 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:57.561116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0383c0-038d-47fa-9e55-846f748808ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:57.561733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0383c0-038d-47fa-9e55-846f748808ff None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:57.562006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0383c0-038d-47fa-9e55-846f748808ff None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:57.562274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0383c0-038d-47fa-9e55-846f748808ff None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:57.562950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0383c0-038d-47fa-9e55-846f748808ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:57.563255 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0383c0-038d-47fa-9e55-846f748808ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:57.564643 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0383c0-038d-47fa-9e55-846f748808ff None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:57.578008 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 355/725] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:57.583521 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2f7be-67ba-4de4-8ee0-dc4b0c7f6e08 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:57.610461 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0383c0-038d-47fa-9e55-846f748808ff None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:57.712116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:57.726189 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2f7be-67ba-4de4-8ee0-dc4b0c7f6e08 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:57.770680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fcf9af-eff7-41a2-ac74-c7a929c490b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:57.771934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcf9af-eff7-41a2-ac74-c7a929c490b5 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:57.773430 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcf9af-eff7-41a2-ac74-c7a929c490b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:57.774420 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcf9af-eff7-41a2-ac74-c7a929c490b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:57.788993 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fcf9af-eff7-41a2-ac74-c7a929c490b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:57.791009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fcf9af-eff7-41a2-ac74-c7a929c490b5 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:57.807021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-b7fcf9af-eff7-41a2-ac74-c7a929c490b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:57.807021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:57.807021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:57.807021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:57.807021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:57.807021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:57.807021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:57.807021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:57.807021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:57.807021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:57.807021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:57.807021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:57.807021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:57.807021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:57.807021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:57.807021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:57.808036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:57.808036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:57.808036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:57.808036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:57.808036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:57.808036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:57.808036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:57.808036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:57.808036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:57.808036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:57.808036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:57.808036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:57.808036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:57.808036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:57.808036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:57.808036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:57.808036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:57.808979 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:57.808979 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:57.808979 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:57.808979 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 371/726] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:57.809678 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabe805-9579-4b4d-acfb-3db5ddf961e7 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:57.809900 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabe805-9579-4b4d-acfb-3db5ddf961e7 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:57.810172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabe805-9579-4b4d-acfb-3db5ddf961e7 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:57.815517 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 372/727] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:57.816358 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabe805-9579-4b4d-acfb-3db5ddf961e7 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:57.824323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0383c0-038d-47fa-9e55-846f748808ff None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:57.860642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:57.867536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f729ae-90e5-44ad-b015-39e3a00551fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:57.868255 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f729ae-90e5-44ad-b015-39e3a00551fe None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:57.868536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f729ae-90e5-44ad-b015-39e3a00551fe None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:57.868808 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f729ae-90e5-44ad-b015-39e3a00551fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:57.881861 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f729ae-90e5-44ad-b015-39e3a00551fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:57.887532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-64f729ae-90e5-44ad-b015-39e3a00551fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:10:57.887532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:57.887532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:57.887532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:57.887532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:57.887532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:57.887532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:57.887532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:57.887532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:57.887532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:57.887532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:57.887532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:57.887532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:57.887532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:57.887532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:57.887532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:57.888573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:57.888573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:57.888573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:57.888573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:57.888573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:57.888573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:57.888573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:57.888573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:57.888573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:57.888573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:57.888573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:57.888573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:57.888573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:57.888573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:57.888573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:57.888573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:57.889396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:57.889396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:57.889396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:57.889396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:57.889396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:57.889396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:57.889396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:57.889396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:57.889396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:57.889396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:57.889396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:10:57.889396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:57.889396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 356/728] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:10:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:57.890635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d317c40-3e18-43bc-a0f7-f243ca5dbfcc None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:57.890635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d317c40-3e18-43bc-a0f7-f243ca5dbfcc None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:57.890899 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d317c40-3e18-43bc-a0f7-f243ca5dbfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:57.910582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-2d317c40-3e18-43bc-a0f7-f243ca5dbfcc None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:57.911900 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2d317c40-3e18-43bc-a0f7-f243ca5dbfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:57.958524 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabe805-9579-4b4d-acfb-3db5ddf961e7 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:57.996027 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 357/729] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:57.997100 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d317c40-3e18-43bc-a0f7-f243ca5dbfcc None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:57.999205 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689abdf-a078-4d9e-9e77-d5b33d66c88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:58.000003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689abdf-a078-4d9e-9e77-d5b33d66c88a None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:58.000380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689abdf-a078-4d9e-9e77-d5b33d66c88a None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:58.000801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689abdf-a078-4d9e-9e77-d5b33d66c88a None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:58.001838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689abdf-a078-4d9e-9e77-d5b33d66c88a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:58.001838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689abdf-a078-4d9e-9e77-d5b33d66c88a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:58.003428 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689abdf-a078-4d9e-9e77-d5b33d66c88a None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:58.023610 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 373/730] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:58.023610 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562efee-16f4-47e1-bd1b-574e8f0f6c0b None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:58.023610 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562efee-16f4-47e1-bd1b-574e8f0f6c0b None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:58.023610 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562efee-16f4-47e1-bd1b-574e8f0f6c0b None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:58.023610 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 374/731] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:58.025538 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562efee-16f4-47e1-bd1b-574e8f0f6c0b None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:58.038918 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:58.120211 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d317c40-3e18-43bc-a0f7-f243ca5dbfcc None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:58.153595 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:58.163223 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562efee-16f4-47e1-bd1b-574e8f0f6c0b None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:58.168391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4faf9-d1aa-42cd-a6a4-7e2cd82e5055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:58.169339 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4faf9-d1aa-42cd-a6a4-7e2cd82e5055 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:58.169845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4faf9-d1aa-42cd-a6a4-7e2cd82e5055 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:58.170333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4faf9-d1aa-42cd-a6a4-7e2cd82e5055 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:58.171296 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4faf9-d1aa-42cd-a6a4-7e2cd82e5055 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:58.171605 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4faf9-d1aa-42cd-a6a4-7e2cd82e5055 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:58.173144 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4faf9-d1aa-42cd-a6a4-7e2cd82e5055 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:58.181858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 358/732] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:10:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:58.182845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d33cc8-e364-4ffe-8a34-b7272926f9f1 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:58.183128 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d33cc8-e364-4ffe-8a34-b7272926f9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:58.183400 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d33cc8-e364-4ffe-8a34-b7272926f9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:58.202121 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d3d33cc8-e364-4ffe-8a34-b7272926f9f1 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:58.203791 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d3d33cc8-e364-4ffe-8a34-b7272926f9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:58.207549 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b9c3a-7725-4c69-a506-6b5f9f2bed9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:58.208404 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b9c3a-7725-4c69-a506-6b5f9f2bed9c None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:58.208746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b9c3a-7725-4c69-a506-6b5f9f2bed9c None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:58.208969 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b9c3a-7725-4c69-a506-6b5f9f2bed9c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:58.217152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b9c3a-7725-4c69-a506-6b5f9f2bed9c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:58.219052 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b9c3a-7725-4c69-a506-6b5f9f2bed9c None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:58.233558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-045b9c3a-7725-4c69-a506-6b5f9f2bed9c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:10:58.233558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:58.233558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:58.233558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:58.233558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:58.233558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:58.233558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:58.233558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:58.233558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:58.233558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:58.233558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:58.233558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:58.233558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:10:58.233558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:10:58.233558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:10:58.233558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:10:58.234525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:58.234525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:58.234525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:10:58.234525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:58.234525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:10:58.234525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:58.234525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:58.234525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:58.234525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:58.234525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:58.234525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:58.234525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:58.234525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:58.234525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:58.234525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:58.234525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:58.234525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:58.234525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:58.235452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:58.235452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:58.235452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:10:58.235452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:10:58.235452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:10:58.235452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:10:58.235452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:10:58.235452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:10:58.235452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:10:58.235452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:58.241577 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 375/733] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:10:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:58.241577 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3aa34-37d3-4e16-92ff-ed14de85badb None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:58.241577 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3aa34-37d3-4e16-92ff-ed14de85badb None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:58.241577 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3aa34-37d3-4e16-92ff-ed14de85badb None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:58.241577 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 376/734] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:10:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:10:58.241577 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3aa34-37d3-4e16-92ff-ed14de85badb None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:58.280289 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 359/735] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:58.281136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d33cc8-e364-4ffe-8a34-b7272926f9f1 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:58.305328 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:58.352879 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3aa34-37d3-4e16-92ff-ed14de85badb None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:58.387225 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fd30d-0ded-44b6-a5f5-b48352be8a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:58.387865 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fd30d-0ded-44b6-a5f5-b48352be8a5a None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:58.388141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fd30d-0ded-44b6-a5f5-b48352be8a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:58.388404 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fd30d-0ded-44b6-a5f5-b48352be8a5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:58.393191 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d33cc8-e364-4ffe-8a34-b7272926f9f1 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:58.400204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77fd30d-0ded-44b6-a5f5-b48352be8a5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:58.401625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77fd30d-0ded-44b6-a5f5-b48352be8a5a None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:58.415592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-d77fd30d-0ded-44b6-a5f5-b48352be8a5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:58.415592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:58.415592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:58.415592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:58.415592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:58.415592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:58.415592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:58.415592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:58.415592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:58.415592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:58.415592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:58.415592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:58.415592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:58.415592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:58.415592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:58.415592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:58.416762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:58.416762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:58.416762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:58.416762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:58.416762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:58.416762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:58.416762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:58.416762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:58.416762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:58.416762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:58.416762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:58.416762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:58.416762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:58.416762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:58.416762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:58.416762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:58.416762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:58.417684 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:58.417684 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:58.417684 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:58.417684 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 377/736] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:58.417917 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1abf6-2263-46e9-94fe-ff65eb6a1774 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:58.418071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1abf6-2263-46e9-94fe-ff65eb6a1774 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:58.418328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1abf6-2263-46e9-94fe-ff65eb6a1774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:58.433999 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:58.435126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710b706-be4c-487d-9f28-e3837a9cd8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:58.435871 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710b706-be4c-487d-9f28-e3837a9cd8af None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:58.436095 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710b706-be4c-487d-9f28-e3837a9cd8af None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:58.436372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710b706-be4c-487d-9f28-e3837a9cd8af None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:58.437195 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710b706-be4c-487d-9f28-e3837a9cd8af None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:58.437479 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710b706-be4c-487d-9f28-e3837a9cd8af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:58.438990 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710b706-be4c-487d-9f28-e3837a9cd8af None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:58.440814 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-23f1abf6-2263-46e9-94fe-ff65eb6a1774 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:10:58.442782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-23f1abf6-2263-46e9-94fe-ff65eb6a1774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:10:58.458206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 360/737] 10.222.0.81 () {56 vars in 1135 bytes} [Mon Sep 19 11:10:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:58.459063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710b706-be4c-487d-9f28-e3837a9cd8af None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:58.518611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 378/738] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:10:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 11:10:58.519681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1abf6-2263-46e9-94fe-ff65eb6a1774 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:58.543693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:58.568948 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710b706-be4c-487d-9f28-e3837a9cd8af None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:58.604764 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe5615-8cb6-458a-89d6-3df7d6fd8bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:58.605359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe5615-8cb6-458a-89d6-3df7d6fd8bae None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:58.605631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe5615-8cb6-458a-89d6-3df7d6fd8bae None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:58.605895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe5615-8cb6-458a-89d6-3df7d6fd8bae None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:58.606535 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe5615-8cb6-458a-89d6-3df7d6fd8bae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:58.607911 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe5615-8cb6-458a-89d6-3df7d6fd8bae None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:58.625415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 361/739] 10.222.0.81 () {60 vars in 1141 bytes} [Mon Sep 19 11:10:58 2022] POST /identity/v3/services => generated 251 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:10:58.626525 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe5615-8cb6-458a-89d6-3df7d6fd8bae None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:58.650255 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1abf6-2263-46e9-94fe-ff65eb6a1774 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:58.713694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:58.721121 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a7c94-5316-4d1a-9131-11ea326f7769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:58.721750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a7c94-5316-4d1a-9131-11ea326f7769 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:58.722123 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a7c94-5316-4d1a-9131-11ea326f7769 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:58.722687 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a7c94-5316-4d1a-9131-11ea326f7769 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:58.724019 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a7c94-5316-4d1a-9131-11ea326f7769 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:58.724597 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a7c94-5316-4d1a-9131-11ea326f7769 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:58.726203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a7c94-5316-4d1a-9131-11ea326f7769 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:58.742108 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 379/740] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:58.743738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a7c94-5316-4d1a-9131-11ea326f7769 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:58.796440 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe5615-8cb6-458a-89d6-3df7d6fd8bae None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:58.836635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2b21f-54b7-4448-adc5-1a823e5eec16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:58.836635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2b21f-54b7-4448-adc5-1a823e5eec16 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:58.836951 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2b21f-54b7-4448-adc5-1a823e5eec16 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:58.837196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2b21f-54b7-4448-adc5-1a823e5eec16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:58.841412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:58.879847 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a7c94-5316-4d1a-9131-11ea326f7769 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:58.914176 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b449e7-fb70-4230-92f8-0bd9fabd2a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:58.914820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2b21f-54b7-4448-adc5-1a823e5eec16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:58.914937 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b449e7-fb70-4230-92f8-0bd9fabd2a93 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:58.915118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b449e7-fb70-4230-92f8-0bd9fabd2a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:58.915390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b449e7-fb70-4230-92f8-0bd9fabd2a93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:58.916325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2b21f-54b7-4448-adc5-1a823e5eec16 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:58.925897 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b449e7-fb70-4230-92f8-0bd9fabd2a93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:10:58.930831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-20b449e7-fb70-4230-92f8-0bd9fabd2a93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:58.930831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:58.930831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:58.930831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:58.930831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:58.930831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:58.930831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:58.930831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:58.930831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:58.930831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:58.930831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:58.930831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:58.930831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:10:58.930831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:10:58.930831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:10:58.930831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:10:58.931813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:10:58.931813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:10:58.931813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:10:58.931813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:10:58.931813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:58.931813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:58.931813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:58.931813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:58.931813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:58.931813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:58.931813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:58.931813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:58.931813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:58.931813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:58.931813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:58.931813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:58.932695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:58.932695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:58.932695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:58.932695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:58.932695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:10:58.932695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:10:58.932695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:10:58.932695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:10:58.932695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:10:58.932695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:10:58.932695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:10:58.932695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:58.932695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 380/741] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:10:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:58.932695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b449e7-fb70-4230-92f8-0bd9fabd2a93 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:58.991203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:59.031890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 362/742] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:10:58 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 11:10:59.032881 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2b21f-54b7-4448-adc5-1a823e5eec16 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:59.048063 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b449e7-fb70-4230-92f8-0bd9fabd2a93 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:59.070053 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:59.086228 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8803340-4023-4a3b-9c4b-c58da72b714a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:59.087415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8803340-4023-4a3b-9c4b-c58da72b714a None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:59.087691 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8803340-4023-4a3b-9c4b-c58da72b714a None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:59.087992 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8803340-4023-4a3b-9c4b-c58da72b714a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:59.095334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8803340-4023-4a3b-9c4b-c58da72b714a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:59.096805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8803340-4023-4a3b-9c4b-c58da72b714a None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:59.108242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-c8803340-4023-4a3b-9c4b-c58da72b714a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:10:59.108242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:59.108242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:59.108242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:59.108242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:59.108242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:59.108242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:59.108242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:59.108242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:59.108242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:59.108242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:59.108242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:59.108242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:10:59.108242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:10:59.108242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:10:59.108242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:10:59.109192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:59.109192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:59.109192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:10:59.109192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:59.109192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:10:59.109192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:10:59.109192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:59.109192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:59.109192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:59.109192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:59.109192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:59.109192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:59.109192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:59.109192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:59.109192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:59.109192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:59.109192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:59.109192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:59.110160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:59.110160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:59.110160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:10:59.110160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:10:59.110160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:10:59.110160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:10:59.110160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:10:59.110160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:10:59.110160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:10:59.110160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:10:59.114042 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 381/743] 10.222.0.81 () {56 vars in 1093 bytes} [Mon Sep 19 11:10:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:59.114876 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8803340-4023-4a3b-9c4b-c58da72b714a None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:59.142685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2b21f-54b7-4448-adc5-1a823e5eec16 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:59.178852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd60e0-db23-4f01-bd19-97dd9448b496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:59.179799 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd60e0-db23-4f01-bd19-97dd9448b496 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:59.180112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd60e0-db23-4f01-bd19-97dd9448b496 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:59.180398 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd60e0-db23-4f01-bd19-97dd9448b496 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:59.181092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd60e0-db23-4f01-bd19-97dd9448b496 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:59.182479 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd60e0-db23-4f01-bd19-97dd9448b496 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:59.198744 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 363/744] 10.222.0.81 () {60 vars in 1141 bytes} [Mon Sep 19 11:10:59 2022] POST /identity/v3/services => generated 240 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:10:59.199561 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd60e0-db23-4f01-bd19-97dd9448b496 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:59.251325 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8803340-4023-4a3b-9c4b-c58da72b714a None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:59.269540 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:59.303861 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9696c4-6fb3-46b5-bd84-434b0d21e32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:59.304950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9696c4-6fb3-46b5-bd84-434b0d21e32e None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:59.305910 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9696c4-6fb3-46b5-bd84-434b0d21e32e None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:59.306373 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9696c4-6fb3-46b5-bd84-434b0d21e32e None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:59.307224 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9696c4-6fb3-46b5-bd84-434b0d21e32e None admin] RBAC: Adding query filter params (name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:59.307712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9696c4-6fb3-46b5-bd84-434b0d21e32e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:59.309387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9696c4-6fb3-46b5-bd84-434b0d21e32e None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:59.319344 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 382/745] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:10:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:59.320609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9696c4-6fb3-46b5-bd84-434b0d21e32e None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:59.323750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:59.332084 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd60e0-db23-4f01-bd19-97dd9448b496 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:59.374126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae574813-2983-4fbb-a270-86ca74266f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:59.374780 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae574813-2983-4fbb-a270-86ca74266f32 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:59.375060 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae574813-2983-4fbb-a270-86ca74266f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:59.375332 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae574813-2983-4fbb-a270-86ca74266f32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:59.388516 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae574813-2983-4fbb-a270-86ca74266f32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:59.389880 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae574813-2983-4fbb-a270-86ca74266f32 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:59.405350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-ae574813-2983-4fbb-a270-86ca74266f32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:59.405350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:59.405350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:59.405350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:59.405350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:59.405350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:59.405350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:59.405350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:59.405350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:59.405350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:59.405350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:59.405350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:59.405350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:59.405350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:59.405350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:59.405350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:59.406327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:59.406327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:59.406327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:59.406327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:59.406327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:59.406327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:59.406327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:59.406327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:59.406327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:59.406327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:59.406327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:59.406327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:59.406327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:59.406327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:59.406327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:59.406327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:59.406327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:59.407401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:59.407401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:59.407401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:59.408100 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 364/746] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:59.408888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae574813-2983-4fbb-a270-86ca74266f32 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:59.447633 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9696c4-6fb3-46b5-bd84-434b0d21e32e None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:59.492116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c7d18-44e0-4bc4-8fd3-96342ded4cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:59.493908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c7d18-44e0-4bc4-8fd3-96342ded4cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:59.494987 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c7d18-44e0-4bc4-8fd3-96342ded4cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:59.495573 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c7d18-44e0-4bc4-8fd3-96342ded4cc4 None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:59.496810 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c7d18-44e0-4bc4-8fd3-96342ded4cc4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:59.497584 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c7d18-44e0-4bc4-8fd3-96342ded4cc4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:59.500162 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c7d18-44e0-4bc4-8fd3-96342ded4cc4 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:59.523252 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 383/747] 10.222.0.81 () {56 vars in 1133 bytes} [Mon Sep 19 11:10:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:59.525243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c7d18-44e0-4bc4-8fd3-96342ded4cc4 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:59.533027 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae574813-2983-4fbb-a270-86ca74266f32 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:59.569118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:59.574247 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697a54c-73c7-4573-8c3a-2b896db3d7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:59.575117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697a54c-73c7-4573-8c3a-2b896db3d7b2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:59.575561 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697a54c-73c7-4573-8c3a-2b896db3d7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:59.576026 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697a54c-73c7-4573-8c3a-2b896db3d7b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:59.589018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1697a54c-73c7-4573-8c3a-2b896db3d7b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:59.590723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1697a54c-73c7-4573-8c3a-2b896db3d7b2 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:59.600287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:59.610190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-1697a54c-73c7-4573-8c3a-2b896db3d7b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:59.610190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:59.610190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:59.610190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:59.610190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:59.610190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:59.610190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:59.610190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:59.610190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:59.610190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:59.610190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:59.610190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:59.610190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:59.610190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:59.610190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:59.610190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:59.611277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:59.611277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:59.611277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:59.611277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:59.611277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:59.611277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:59.611277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:59.611277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:59.611277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:59.611277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:59.611277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:59.611277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:59.611277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:59.611277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:59.611277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:59.611277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:59.611277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:59.612932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:59.612932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:59.612932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:59.614891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 365/748] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:59.616362 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697a54c-73c7-4573-8c3a-2b896db3d7b2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:59.662282 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c7d18-44e0-4bc4-8fd3-96342ded4cc4 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:59.703911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b0899-bc3c-4bcb-aee3-a4fd9880d3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:59.704879 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b0899-bc3c-4bcb-aee3-a4fd9880d3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:59.705365 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b0899-bc3c-4bcb-aee3-a4fd9880d3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:59.705839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b0899-bc3c-4bcb-aee3-a4fd9880d3c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:59.706962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b0899-bc3c-4bcb-aee3-a4fd9880d3c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:59.708180 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b0899-bc3c-4bcb-aee3-a4fd9880d3c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:59.709563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b0899-bc3c-4bcb-aee3-a4fd9880d3c0 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:59.724682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 384/749] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:59.725926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b0899-bc3c-4bcb-aee3-a4fd9880d3c0 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:59.749191 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697a54c-73c7-4573-8c3a-2b896db3d7b2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:59.796420 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4fb2d-a220-46e2-9c31-716e099a6354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:59.797011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4fb2d-a220-46e2-9c31-716e099a6354 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:59.797740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4fb2d-a220-46e2-9c31-716e099a6354 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:59.797740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4fb2d-a220-46e2-9c31-716e099a6354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:59.813732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4fb2d-a220-46e2-9c31-716e099a6354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:59.815522 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4fb2d-a220-46e2-9c31-716e099a6354 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:59.837024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-52d4fb2d-a220-46e2-9c31-716e099a6354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:59.837024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:10:59.837024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:10:59.837024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:10:59.837024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:10:59.837024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:10:59.837024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:10:59.837024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:10:59.837024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:10:59.837024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:10:59.837024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:10:59.837024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:10:59.837024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:10:59.837024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:10:59.837024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:10:59.837024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:10:59.838862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:10:59.838862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:10:59.838862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:10:59.838862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:10:59.838862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:10:59.838862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:10:59.838862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:10:59.838862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:10:59.838862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:10:59.838862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:10:59.838862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:10:59.838862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:10:59.838862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:10:59.838862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:10:59.838862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:10:59.838862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:10:59.838862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:10:59.840003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:10:59.840003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:10:59.840003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:10:59.840003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 366/750] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:10:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:10:59.840003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4fb2d-a220-46e2-9c31-716e099a6354 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:59.849669 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:59.867348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b0899-bc3c-4bcb-aee3-a4fd9880d3c0 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:10:59.911126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ac686-e90a-4342-8716-a28f2000e30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:10:59.912050 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ac686-e90a-4342-8716-a28f2000e30a None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:10:59.912505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ac686-e90a-4342-8716-a28f2000e30a None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:10:59.913538 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:10:59.913933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ac686-e90a-4342-8716-a28f2000e30a None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:10:59.915468 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ac686-e90a-4342-8716-a28f2000e30a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:10:59.915954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ac686-e90a-4342-8716-a28f2000e30a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:10:59.917644 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ac686-e90a-4342-8716-a28f2000e30a None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:10:59.934904 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 385/751] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:10:59.936185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ac686-e90a-4342-8716-a28f2000e30a None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:10:59.974010 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4fb2d-a220-46e2-9c31-716e099a6354 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:00.017749 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279867c-1afb-41bf-b1a3-079eb792a8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:00.018551 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279867c-1afb-41bf-b1a3-079eb792a8f8 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:00.018933 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279867c-1afb-41bf-b1a3-079eb792a8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:00.019538 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279867c-1afb-41bf-b1a3-079eb792a8f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:00.028467 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279867c-1afb-41bf-b1a3-079eb792a8f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:00.030450 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279867c-1afb-41bf-b1a3-079eb792a8f8 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:00.045750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-c279867c-1afb-41bf-b1a3-079eb792a8f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:00.045750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:00.045750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:00.045750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:00.045750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:00.045750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:00.045750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:00.045750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:00.045750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:00.045750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:00.045750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:00.045750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:00.045750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:11:00.045750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:11:00.045750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:11:00.046974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:11:00.046974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:00.046974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:00.046974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:00.046974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:00.046974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:00.046974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:00.046974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:00.046974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:00.046974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:00.046974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:00.046974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:00.046974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:00.046974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:00.046974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:00.046974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:00.046974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:00.048917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:11:00.048917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:11:00.048917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:11:00.048917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:11:00.048917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:11:00.048917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:11:00.048917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:00.048917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:00.048917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 367/752] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:10:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:00.048917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279867c-1afb-41bf-b1a3-079eb792a8f8 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:00.059928 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ac686-e90a-4342-8716-a28f2000e30a None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:00.102036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a854d-1215-4a7f-8bc8-477d81e25518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:00.104309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a854d-1215-4a7f-8bc8-477d81e25518 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:00.104924 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a854d-1215-4a7f-8bc8-477d81e25518 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:00.105407 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a854d-1215-4a7f-8bc8-477d81e25518 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:00.106532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a854d-1215-4a7f-8bc8-477d81e25518 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:00.107530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a854d-1215-4a7f-8bc8-477d81e25518 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:00.109572 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a854d-1215-4a7f-8bc8-477d81e25518 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:00.123698 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 386/753] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:00.125536 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a854d-1215-4a7f-8bc8-477d81e25518 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:00.139530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:00.172429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279867c-1afb-41bf-b1a3-079eb792a8f8 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:00.187677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:00.212593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53f628-46db-464b-8c0d-97ab0d5114bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:00.213320 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53f628-46db-464b-8c0d-97ab0d5114bc None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:00.213802 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53f628-46db-464b-8c0d-97ab0d5114bc None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:00.214300 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53f628-46db-464b-8c0d-97ab0d5114bc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:00.228124 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53f628-46db-464b-8c0d-97ab0d5114bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:00.230328 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53f628-46db-464b-8c0d-97ab0d5114bc None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:00.252360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-1e53f628-46db-464b-8c0d-97ab0d5114bc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:11:00.252360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:00.252360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:00.252360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:00.252360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:00.252360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:00.252360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:00.252360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:00.252360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:00.252360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:00.252360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:00.252360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:00.252360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:11:00.252360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:11:00.252360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:11:00.252360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:11:00.253370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:00.253370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:00.253370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:11:00.253370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:00.253370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:11:00.253370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:00.253370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:00.253370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:00.253370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:00.253370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:00.253370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:00.253370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:00.253370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:00.253370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:00.253370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:00.253370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:00.253370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:00.253370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:00.254335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:00.254335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:00.254335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:11:00.254335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:11:00.254335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:11:00.254335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:11:00.254335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:11:00.254335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:11:00.254335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:11:00.254335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:00.254335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 368/754] 10.222.0.81 () {56 vars in 1117 bytes} [Mon Sep 19 11:11:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:00.254955 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53f628-46db-464b-8c0d-97ab0d5114bc None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:00.260709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a854d-1215-4a7f-8bc8-477d81e25518 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:00.312619 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287b3b0-eefe-4e17-82ec-6e0548345c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:00.313229 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287b3b0-eefe-4e17-82ec-6e0548345c0e None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:00.313512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287b3b0-eefe-4e17-82ec-6e0548345c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:00.313987 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287b3b0-eefe-4e17-82ec-6e0548345c0e None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:00.315803 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7287b3b0-eefe-4e17-82ec-6e0548345c0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:00.316120 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7287b3b0-eefe-4e17-82ec-6e0548345c0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:00.317843 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7287b3b0-eefe-4e17-82ec-6e0548345c0e None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:00.337240 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 387/755] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:11:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:00.338186 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287b3b0-eefe-4e17-82ec-6e0548345c0e None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:00.376532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53f628-46db-464b-8c0d-97ab0d5114bc None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:00.419939 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15c965-c05e-4404-bb9c-a1c8d60bc45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:00.421091 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15c965-c05e-4404-bb9c-a1c8d60bc45c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:00.421578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15c965-c05e-4404-bb9c-a1c8d60bc45c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:00.423066 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15c965-c05e-4404-bb9c-a1c8d60bc45c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:00.430769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce15c965-c05e-4404-bb9c-a1c8d60bc45c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:00.432536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce15c965-c05e-4404-bb9c-a1c8d60bc45c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:00.444683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-ce15c965-c05e-4404-bb9c-a1c8d60bc45c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:00.444683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:00.444683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:00.444683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:00.444683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:00.444683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:00.444683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:00.444683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:00.444683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:00.444683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:00.444683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:00.444683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:00.444683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:11:00.444683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:11:00.444683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:11:00.445615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:11:00.445615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:00.445615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:00.445615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:00.445615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:00.445615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:00.445615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:00.445615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:00.445615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:00.445615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:00.445615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:00.445615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:00.445615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:00.445615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:00.445615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:00.445615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:00.445615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:00.447062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:11:00.447062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:11:00.447062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:11:00.447062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:11:00.447062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:11:00.447062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:11:00.447062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:00.447062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:00.449564 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:00.450935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 369/756] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:11:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:00.452015 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15c965-c05e-4404-bb9c-a1c8d60bc45c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:00.487489 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:00.494181 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287b3b0-eefe-4e17-82ec-6e0548345c0e None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:00.539734 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20efdb3-de9b-4736-a4c3-7a44b54258cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:00.540358 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20efdb3-de9b-4736-a4c3-7a44b54258cf None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:00.540640 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20efdb3-de9b-4736-a4c3-7a44b54258cf None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:00.540917 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20efdb3-de9b-4736-a4c3-7a44b54258cf None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:00.541746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20efdb3-de9b-4736-a4c3-7a44b54258cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:00.542058 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20efdb3-de9b-4736-a4c3-7a44b54258cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:00.543911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20efdb3-de9b-4736-a4c3-7a44b54258cf None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:00.567710 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 388/757] 10.222.0.81 () {56 vars in 1149 bytes} [Mon Sep 19 11:11:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:00.568596 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20efdb3-de9b-4736-a4c3-7a44b54258cf None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:00.568904 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15c965-c05e-4404-bb9c-a1c8d60bc45c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:00.619962 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12ae50-afb5-4b97-8752-a5b9f415da62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:00.620577 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12ae50-afb5-4b97-8752-a5b9f415da62 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:00.620860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12ae50-afb5-4b97-8752-a5b9f415da62 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:00.621137 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12ae50-afb5-4b97-8752-a5b9f415da62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:00.621874 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12ae50-afb5-4b97-8752-a5b9f415da62 None admin] RBAC: Adding query filter params (role.id=753f8b35d5f0458d8f62d0ad6e28033f, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=1c8a9732807d4921a41a6a2d5019e2f5) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:00.622190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12ae50-afb5-4b97-8752-a5b9f415da62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:00.624265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12ae50-afb5-4b97-8752-a5b9f415da62 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:00.634308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 370/758] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:11:00 2022] GET /identity/v3/role_assignments?user.id=1c8a9732807d4921a41a6a2d5019e2f5&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=753f8b35d5f0458d8f62d0ad6e28033f => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:00.635575 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12ae50-afb5-4b97-8752-a5b9f415da62 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:00.765295 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20efdb3-de9b-4736-a4c3-7a44b54258cf None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:00.839144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:00.840973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:00.849543 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12ae50-afb5-4b97-8752-a5b9f415da62 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:00.861199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62ed4c-8b33-494a-89cd-dbdcbc61aab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:00.861821 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62ed4c-8b33-494a-89cd-dbdcbc61aab0 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:00.862187 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62ed4c-8b33-494a-89cd-dbdcbc61aab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:00.862463 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62ed4c-8b33-494a-89cd-dbdcbc61aab0 None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:00.863153 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62ed4c-8b33-494a-89cd-dbdcbc61aab0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:00.863558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62ed4c-8b33-494a-89cd-dbdcbc61aab0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:00.866404 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62ed4c-8b33-494a-89cd-dbdcbc61aab0 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:00.890505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 389/759] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:11:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:00.896056 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62ed4c-8b33-494a-89cd-dbdcbc61aab0 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:00.910732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f078dd-6ee1-439f-a0fe-55d7c54e72af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:00.911351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f078dd-6ee1-439f-a0fe-55d7c54e72af None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:00.911633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f078dd-6ee1-439f-a0fe-55d7c54e72af None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:00.911946 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f078dd-6ee1-439f-a0fe-55d7c54e72af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:01.002258 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f078dd-6ee1-439f-a0fe-55d7c54e72af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:01.004263 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f078dd-6ee1-439f-a0fe-55d7c54e72af None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:01.023165 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62ed4c-8b33-494a-89cd-dbdcbc61aab0 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:01.065196 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd988e-d61a-43be-891f-8922f6d57a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:01.066342 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd988e-d61a-43be-891f-8922f6d57a8a None admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:01.066884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd988e-d61a-43be-891f-8922f6d57a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:01.067383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd988e-d61a-43be-891f-8922f6d57a8a None admin] PATH_INFO: `/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/5abd70a18ccd407bafe3b61c8870986e/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:01.069823 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd988e-d61a-43be-891f-8922f6d57a8a None admin] RBAC: Authorizing `identity:create_grant(project_id=ed9a0b626e3e4b93b391158e981ec586, user_id=5abd70a18ccd407bafe3b61c8870986e, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:01.071700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd988e-d61a-43be-891f-8922f6d57a8a None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:01.090748 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 390/760] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:11:00 2022] PUT /identity/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/5abd70a18ccd407bafe3b61c8870986e/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:11:01.092510 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89a5b5-03a3-47ea-9144-82761e99c044 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:01.093045 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89a5b5-03a3-47ea-9144-82761e99c044 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:01.093651 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89a5b5-03a3-47ea-9144-82761e99c044 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:01.099966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 391/761] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:01.101774 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba77f82-f71b-4b86-8fe4-2c1004c1d26a None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:01.102872 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba77f82-f71b-4b86-8fe4-2c1004c1d26a None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:01.103344 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba77f82-f71b-4b86-8fe4-2c1004c1d26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:01.125739 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-7ba77f82-f71b-4b86-8fe4-2c1004c1d26a None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:01.127476 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-7ba77f82-f71b-4b86-8fe4-2c1004c1d26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:01.157549 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:01.168740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 371/762] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:11:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:11:01.173864 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f078dd-6ee1-439f-a0fe-55d7c54e72af None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:01.189254 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:01.217756 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 392/763] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:01.327013 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f078dd-6ee1-439f-a0fe-55d7c54e72af None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:01.367244 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076ad36-12c3-4432-9121-3f2576ac1811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:01.368514 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076ad36-12c3-4432-9121-3f2576ac1811 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:01.368793 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076ad36-12c3-4432-9121-3f2576ac1811 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:01.369069 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076ad36-12c3-4432-9121-3f2576ac1811 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:01.380258 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076ad36-12c3-4432-9121-3f2576ac1811 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:01.381728 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076ad36-12c3-4432-9121-3f2576ac1811 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:01.398221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-1076ad36-12c3-4432-9121-3f2576ac1811 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:01.398221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:01.398221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:01.398221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:01.398221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:01.398221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:01.398221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:01.398221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:01.398221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:01.398221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:01.398221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:01.398221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:01.398221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:01.398221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:01.398221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:01.398221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:01.399246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:01.399246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:01.399246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:01.399246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:01.399246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:01.399246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:01.399246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:01.399246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:01.399246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:01.399246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:01.399246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:01.399246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:01.399246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:01.399246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:01.399246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:01.399246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:01.399246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:01.400166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:01.400166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:01.400166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:01.400166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 372/764] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:01.404798 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba77f82-f71b-4b86-8fe4-2c1004c1d26a None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:01.519667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba77f82-f71b-4b86-8fe4-2c1004c1d26a None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:01.537835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076ad36-12c3-4432-9121-3f2576ac1811 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:01.565373 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790210a-3182-4ccb-b089-3f84b174ac60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:01.566404 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790210a-3182-4ccb-b089-3f84b174ac60 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:01.567442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790210a-3182-4ccb-b089-3f84b174ac60 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:01.568061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790210a-3182-4ccb-b089-3f84b174ac60 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:01.569627 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790210a-3182-4ccb-b089-3f84b174ac60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:01.570332 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790210a-3182-4ccb-b089-3f84b174ac60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:01.574097 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790210a-3182-4ccb-b089-3f84b174ac60 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:01.597133 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 393/765] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:01.600204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790210a-3182-4ccb-b089-3f84b174ac60 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:01.647302 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:01.660757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:01.722204 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076ad36-12c3-4432-9121-3f2576ac1811 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:01.733802 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790210a-3182-4ccb-b089-3f84b174ac60 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:01.769916 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b57996-ab0e-4a92-92da-689416a10d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:01.770602 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b57996-ab0e-4a92-92da-689416a10d63 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:01.770886 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b57996-ab0e-4a92-92da-689416a10d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:01.771159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b57996-ab0e-4a92-92da-689416a10d63 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:01.771913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b57996-ab0e-4a92-92da-689416a10d63 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:01.773543 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b57996-ab0e-4a92-92da-689416a10d63 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:01.775293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db854d0-401b-486c-b50c-4dfe48b2754d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:01.776292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db854d0-401b-486c-b50c-4dfe48b2754d None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:01.776887 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db854d0-401b-486c-b50c-4dfe48b2754d None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:01.777361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db854d0-401b-486c-b50c-4dfe48b2754d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:01.784410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-57b57996-ab0e-4a92-92da-689416a10d63 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:11:01.784410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:01.784410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:01.784410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:01.784410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:01.784410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:01.784410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:01.784410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:01.784410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:11:01.784410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:01.784410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:01.784410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:01.784410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:01.784410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:01.784410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:01.784410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:01.784410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:01.785432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:01.785432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:01.785432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:01.785432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:01.785432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:01.785432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:01.785432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:01.785432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:01.785432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:01.785432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:01.785432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:01.785432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:01.785432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:01.785432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:01.785432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:01.785432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:01.786961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:01.786961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:01.786961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:01.786961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:01.786961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:01.786961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:01.786961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:01.786961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:01.786961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:01.786961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:01.786961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:01.786961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:01.786961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:11:01.786961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:01.786961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 373/766] 10.222.0.81 () {56 vars in 1129 bytes} [Mon Sep 19 11:11:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:01.787845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63516e0a-086e-4815-9d22-bedb83b51d33 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:01.787845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63516e0a-086e-4815-9d22-bedb83b51d33 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:01.787845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63516e0a-086e-4815-9d22-bedb83b51d33 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:01.789028 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db854d0-401b-486c-b50c-4dfe48b2754d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:01.789290 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 374/767] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:01.793404 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63516e0a-086e-4815-9d22-bedb83b51d33 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:01.795199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db854d0-401b-486c-b50c-4dfe48b2754d None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:01.809453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-1db854d0-401b-486c-b50c-4dfe48b2754d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:01.809453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:01.809453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:01.809453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:01.809453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:01.809453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:01.809453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:01.809453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:01.809453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:01.809453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:01.809453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:01.809453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:01.809453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:11:01.809453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:11:01.809453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:11:01.810385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:11:01.810385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:01.810385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:01.810385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:01.810385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:01.810385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:01.810385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:01.810385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:01.810385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:01.810385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:01.810385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:01.810385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:01.810385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:01.810385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:01.810385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:01.810385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:01.810385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:01.811313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:11:01.811313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:11:01.811313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:11:01.811313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:11:01.811313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:11:01.811313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:11:01.811313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:01.811313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:01.813895 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 394/768] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:11:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:01.815371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1151f-adfc-43e5-9987-dd2466564a07 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:01.815863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1151f-adfc-43e5-9987-dd2466564a07 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:01.816363 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1151f-adfc-43e5-9987-dd2466564a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:01.836604 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-f1f1151f-adfc-43e5-9987-dd2466564a07 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:01.838068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f1f1151f-adfc-43e5-9987-dd2466564a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:01.916732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63516e0a-086e-4815-9d22-bedb83b51d33 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:01.917396 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 395/769] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:01.918851 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f1151f-adfc-43e5-9987-dd2466564a07 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:01.932159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:01.957231 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced10910-f04d-4d99-a507-edda3ff824da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:01.957951 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced10910-f04d-4d99-a507-edda3ff824da None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:01.958291 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced10910-f04d-4d99-a507-edda3ff824da None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:01.958635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced10910-f04d-4d99-a507-edda3ff824da None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:01.959589 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced10910-f04d-4d99-a507-edda3ff824da None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:01.959986 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced10910-f04d-4d99-a507-edda3ff824da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:01.961866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced10910-f04d-4d99-a507-edda3ff824da None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:01.967537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 375/770] 10.222.0.81 () {56 vars in 1124 bytes} [Mon Sep 19 11:11:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:01.972644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced10910-f04d-4d99-a507-edda3ff824da None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:01.983086 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:02.136029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f1151f-adfc-43e5-9987-dd2466564a07 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:02.140098 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced10910-f04d-4d99-a507-edda3ff824da None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:02.186168 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5897a-4907-4abb-b2fa-0a1ac6d00012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:02.186700 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5897a-4907-4abb-b2fa-0a1ac6d00012 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:02.187594 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5897a-4907-4abb-b2fa-0a1ac6d00012 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:02.188027 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5897a-4907-4abb-b2fa-0a1ac6d00012 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:02.188731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5897a-4907-4abb-b2fa-0a1ac6d00012 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:02.189044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5897a-4907-4abb-b2fa-0a1ac6d00012 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:02.190467 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5897a-4907-4abb-b2fa-0a1ac6d00012 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:02.196905 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 376/771] 10.222.0.81 () {56 vars in 1124 bytes} [Mon Sep 19 11:11:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:02.202815 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5897a-4907-4abb-b2fa-0a1ac6d00012 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:02.241072 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e94cfe9-dd0b-41f7-b582-77c54f16e11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:02.242047 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94cfe9-dd0b-41f7-b582-77c54f16e11d None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:02.242328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94cfe9-dd0b-41f7-b582-77c54f16e11d None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:02.243204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94cfe9-dd0b-41f7-b582-77c54f16e11d None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:02.243934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e94cfe9-dd0b-41f7-b582-77c54f16e11d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:02.244245 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e94cfe9-dd0b-41f7-b582-77c54f16e11d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:02.245667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e94cfe9-dd0b-41f7-b582-77c54f16e11d None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:02.254295 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:02.264720 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 396/772] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:11:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:02.265810 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e94cfe9-dd0b-41f7-b582-77c54f16e11d None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:02.329081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5897a-4907-4abb-b2fa-0a1ac6d00012 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:02.372228 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c3152-ea27-4f23-81fc-570392184556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:02.372853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c3152-ea27-4f23-81fc-570392184556 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:02.373131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c3152-ea27-4f23-81fc-570392184556 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:02.373401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c3152-ea27-4f23-81fc-570392184556 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:02.374094 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c3152-ea27-4f23-81fc-570392184556 None admin] RBAC: Adding query filter params (interface=public, service_id=f14c023452144bb68750fc896f7c415b, region_id=RegionOne) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:02.374399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c3152-ea27-4f23-81fc-570392184556 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:02.375854 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c3152-ea27-4f23-81fc-570392184556 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:02.386351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 377/773] 10.222.0.81 () {56 vars in 1251 bytes} [Mon Sep 19 11:11:02 2022] GET /identity/v3/endpoints?service_id=f14c023452144bb68750fc896f7c415b&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:02.387366 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c3152-ea27-4f23-81fc-570392184556 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:02.388802 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:02.413159 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e94cfe9-dd0b-41f7-b582-77c54f16e11d None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:02.459147 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14b0e4-d239-412a-a636-23e4f19266bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:02.460576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14b0e4-d239-412a-a636-23e4f19266bd None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:02.461356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14b0e4-d239-412a-a636-23e4f19266bd None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:02.461841 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14b0e4-d239-412a-a636-23e4f19266bd None admin] PATH_INFO: `/v3/services/network` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:02.463210 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f14b0e4-d239-412a-a636-23e4f19266bd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:02.465143 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f14b0e4-d239-412a-a636-23e4f19266bd None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:02.476222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-8f14b0e4-d239-412a-a636-23e4f19266bd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:11:02.476222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:02.476222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:02.476222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:02.476222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:02.476222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:02.476222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:02.476222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:02.476222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:11:02.476222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:02.476222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:02.476222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:02.476222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:02.476222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:02.476222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:02.476222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:02.476222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:02.477296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:02.477296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:02.477296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:02.477296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:02.477296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:02.477296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:02.477296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:02.477296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:02.477296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:02.477296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:02.477296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:02.477296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:02.477296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:02.477296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:02.477296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:02.477296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:02.483111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:02.483111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:02.483111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:02.483111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:02.483111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:02.483111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:02.483111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:02.483111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:02.483111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:02.483111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:02.483111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:02.483111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:02.483111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:11:02.483111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:02.483111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 397/774] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:11:02 2022] GET /identity/v3/services/network => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:02.483986 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14b0e4-d239-412a-a636-23e4f19266bd None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:02.518278 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c3152-ea27-4f23-81fc-570392184556 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:02.521227 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:02.559998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bd871-4687-43d2-aaa8-9705ba935c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:02.561014 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bd871-4687-43d2-aaa8-9705ba935c0d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:02.561668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bd871-4687-43d2-aaa8-9705ba935c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:02.562173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bd871-4687-43d2-aaa8-9705ba935c0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:02.563157 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4bd871-4687-43d2-aaa8-9705ba935c0d None admin] RBAC: Adding query filter params (role.id=af9d39307d714527b14b2045ab466e7a, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=06d279b488d148428b7a3c35a9786ffe) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:02.563696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4bd871-4687-43d2-aaa8-9705ba935c0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:02.565501 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4bd871-4687-43d2-aaa8-9705ba935c0d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:02.572508 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 378/775] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:11:02 2022] GET /identity/v3/role_assignments?user.id=06d279b488d148428b7a3c35a9786ffe&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=af9d39307d714527b14b2045ab466e7a => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:02.573605 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bd871-4687-43d2-aaa8-9705ba935c0d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:02.592028 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14b0e4-d239-412a-a636-23e4f19266bd None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:02.683222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46742c33-9db4-48cd-acf8-51be060fc87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:02.684274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46742c33-9db4-48cd-acf8-51be060fc87c None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:02.684935 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46742c33-9db4-48cd-acf8-51be060fc87c None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:02.685436 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46742c33-9db4-48cd-acf8-51be060fc87c None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:02.686407 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46742c33-9db4-48cd-acf8-51be060fc87c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:02.689506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46742c33-9db4-48cd-acf8-51be060fc87c None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:02.702925 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 398/776] 10.222.0.81 () {56 vars in 1087 bytes} [Mon Sep 19 11:11:02 2022] GET /identity/v3/services => generated 772 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:02.871368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bd871-4687-43d2-aaa8-9705ba935c0d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:02.910905 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3d805-c74d-4cae-b265-82c46de0d0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:02.912015 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3d805-c74d-4cae-b265-82c46de0d0aa None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:02.912524 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3d805-c74d-4cae-b265-82c46de0d0aa None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:02.913060 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3d805-c74d-4cae-b265-82c46de0d0aa None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:02.913903 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3d805-c74d-4cae-b265-82c46de0d0aa None admin] RBAC: Adding query filter params (name=network) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:02.914943 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3d805-c74d-4cae-b265-82c46de0d0aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:02.916557 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3d805-c74d-4cae-b265-82c46de0d0aa None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:02.926620 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 379/777] 10.222.0.81 () {56 vars in 1112 bytes} [Mon Sep 19 11:11:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:02.928922 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46742c33-9db4-48cd-acf8-51be060fc87c None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:02.951937 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:03.068288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46742c33-9db4-48cd-acf8-51be060fc87c None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:03.104419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f05e5d-3a4f-4fa8-805d-328015509bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:03.105416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f05e5d-3a4f-4fa8-805d-328015509bfd None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:03.105929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f05e5d-3a4f-4fa8-805d-328015509bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:03.106420 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f05e5d-3a4f-4fa8-805d-328015509bfd None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:03.107255 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f05e5d-3a4f-4fa8-805d-328015509bfd None admin] RBAC: Adding query filter params (type=network) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:03.107776 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f05e5d-3a4f-4fa8-805d-328015509bfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:03.109346 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f05e5d-3a4f-4fa8-805d-328015509bfd None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:03.118927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 399/778] 10.222.0.81 () {56 vars in 1112 bytes} [Mon Sep 19 11:11:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:03.124861 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3d805-c74d-4cae-b265-82c46de0d0aa None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:03.166331 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:03.262170 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3d805-c74d-4cae-b265-82c46de0d0aa None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:03.263585 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05678e56-3ce4-4be4-8eda-8bb42ab8de69 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:03.263926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05678e56-3ce4-4be4-8eda-8bb42ab8de69 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:03.264206 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05678e56-3ce4-4be4-8eda-8bb42ab8de69 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:03.265986 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 400/779] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:11:03.271260 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79a0c8-0ee3-47c9-ac37-0bbd58b8727e None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:03.271549 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79a0c8-0ee3-47c9-ac37-0bbd58b8727e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:03.271970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79a0c8-0ee3-47c9-ac37-0bbd58b8727e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:03.302056 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-af79a0c8-0ee3-47c9-ac37-0bbd58b8727e None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:03.303385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-af79a0c8-0ee3-47c9-ac37-0bbd58b8727e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:03.315180 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76686b9d-fae3-4c08-8581-84a61bf1215a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:03.315822 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76686b9d-fae3-4c08-8581-84a61bf1215a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:03.316108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76686b9d-fae3-4c08-8581-84a61bf1215a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:03.316375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76686b9d-fae3-4c08-8581-84a61bf1215a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:03.317071 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76686b9d-fae3-4c08-8581-84a61bf1215a None admin] RBAC: Adding query filter params (interface=public, service_id=e4194509f1444d779b9dd7974b055e3b, region_id=RegionOne) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:03.317374 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76686b9d-fae3-4c08-8581-84a61bf1215a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:03.318852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76686b9d-fae3-4c08-8581-84a61bf1215a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:03.324343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 380/780] 10.222.0.81 () {56 vars in 1251 bytes} [Mon Sep 19 11:11:03 2022] GET /identity/v3/endpoints?service_id=e4194509f1444d779b9dd7974b055e3b&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:03.330106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76686b9d-fae3-4c08-8581-84a61bf1215a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:03.450614 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 401/781] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:11:03.478876 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:03.486756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76686b9d-fae3-4c08-8581-84a61bf1215a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:03.523500 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd68ef-5063-4d0f-90f4-4ee46cd51b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:03.524696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd68ef-5063-4d0f-90f4-4ee46cd51b3f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:03.524975 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd68ef-5063-4d0f-90f4-4ee46cd51b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:03.525247 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd68ef-5063-4d0f-90f4-4ee46cd51b3f None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:03.525850 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd68ef-5063-4d0f-90f4-4ee46cd51b3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:03.527221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd68ef-5063-4d0f-90f4-4ee46cd51b3f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:03.533435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 381/782] 10.222.0.81 () {56 vars in 1087 bytes} [Mon Sep 19 11:11:03 2022] GET /identity/v3/services => generated 772 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:03.588374 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33bc73-ae90-45b0-8de5-b12e745bb263 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:03.588980 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33bc73-ae90-45b0-8de5-b12e745bb263 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:03.589449 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33bc73-ae90-45b0-8de5-b12e745bb263 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:03.591254 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 402/783] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:03.596727 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c5ab4-4f1d-413a-9fe6-33b15bb84aa5 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:03.597163 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c5ab4-4f1d-413a-9fe6-33b15bb84aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:03.597441 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c5ab4-4f1d-413a-9fe6-33b15bb84aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:03.617151 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-733c5ab4-4f1d-413a-9fe6-33b15bb84aa5 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:03.618442 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-733c5ab4-4f1d-413a-9fe6-33b15bb84aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:03.698436 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 382/784] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:03.777839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33bc73-ae90-45b0-8de5-b12e745bb263 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:03.788956 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:03.888640 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33bc73-ae90-45b0-8de5-b12e745bb263 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:03.927021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc57322-3107-47cc-b0bb-6770998e65c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:03.927656 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc57322-3107-47cc-b0bb-6770998e65c6 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:03.927963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc57322-3107-47cc-b0bb-6770998e65c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:03.928235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc57322-3107-47cc-b0bb-6770998e65c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:03.939026 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc57322-3107-47cc-b0bb-6770998e65c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:11:03.944107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-4bc57322-3107-47cc-b0bb-6770998e65c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:11:03.944107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:03.944107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:03.944107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:03.944107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:03.944107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:03.944107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:03.944107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:03.944107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:03.944107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:03.944107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:03.944107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:03.944107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:11:03.944107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:11:03.944107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:11:03.944107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:11:03.945044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:11:03.945044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:11:03.945044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:11:03.945044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:11:03.945044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:03.945044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:03.945044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:03.945044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:03.945044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:03.945044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:03.945044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:03.945044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:03.945044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:03.945044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:03.945044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:03.945044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:03.945934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:03.945934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:03.945934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:03.945934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:03.945934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:11:03.945934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:11:03.945934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:11:03.945934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:11:03.945934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:11:03.945934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:11:03.945934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:11:03.945934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:03.945934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 403/785] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:11:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:03.951559 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c5ab4-4f1d-413a-9fe6-33b15bb84aa5 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:04.001279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc57322-3107-47cc-b0bb-6770998e65c6 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:04.096142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:04.115450 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c5ab4-4f1d-413a-9fe6-33b15bb84aa5 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:04.129959 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:04.185004 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc57322-3107-47cc-b0bb-6770998e65c6 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:04.187376 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db90360-dcc5-4fb0-bf03-bc83fc142331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:04.188209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db90360-dcc5-4fb0-bf03-bc83fc142331 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:04.188637 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db90360-dcc5-4fb0-bf03-bc83fc142331 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:04.189073 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db90360-dcc5-4fb0-bf03-bc83fc142331 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:04.189900 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db90360-dcc5-4fb0-bf03-bc83fc142331 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:04.191016 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db90360-dcc5-4fb0-bf03-bc83fc142331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:04.192579 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db90360-dcc5-4fb0-bf03-bc83fc142331 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:04.213559 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 383/786] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:11:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:04.220691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db90360-dcc5-4fb0-bf03-bc83fc142331 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:04.266250 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40123a3-8866-43a4-a022-c4427581253e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:04.267224 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40123a3-8866-43a4-a022-c4427581253e None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:04.267673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40123a3-8866-43a4-a022-c4427581253e None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:04.268317 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40123a3-8866-43a4-a022-c4427581253e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:04.287338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40123a3-8866-43a4-a022-c4427581253e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:11:04.293496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-b40123a3-8866-43a4-a022-c4427581253e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:11:04.293496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:04.293496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:04.293496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:04.293496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:04.293496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:04.293496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:04.293496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:04.293496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:04.293496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:04.293496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:04.293496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:04.293496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:11:04.293496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:11:04.293496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:11:04.293496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:11:04.294657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:11:04.294657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:11:04.294657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:11:04.294657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:11:04.294657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:04.294657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:04.294657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:04.294657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:04.294657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:04.294657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:04.294657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:04.294657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:04.294657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:04.294657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:04.294657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:04.294657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:04.295724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:04.295724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:04.295724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:04.295724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:04.295724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:11:04.295724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:11:04.295724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:11:04.295724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:11:04.295724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:11:04.295724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:11:04.295724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:11:04.295724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:04.297997 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 404/787] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:11:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:04.304287 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40123a3-8866-43a4-a022-c4427581253e None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:04.426138 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db90360-dcc5-4fb0-bf03-bc83fc142331 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:04.489385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40123a3-8866-43a4-a022-c4427581253e None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:04.493272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65721387-0353-45d8-83d0-231f2852dce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:04.494154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65721387-0353-45d8-83d0-231f2852dce6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:04.494582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65721387-0353-45d8-83d0-231f2852dce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:04.495012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65721387-0353-45d8-83d0-231f2852dce6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:04.510918 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65721387-0353-45d8-83d0-231f2852dce6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:04.512823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65721387-0353-45d8-83d0-231f2852dce6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:04.529689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-65721387-0353-45d8-83d0-231f2852dce6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:04.529689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:04.529689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:04.529689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:04.529689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:04.529689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:04.529689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:04.529689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:04.529689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:04.529689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:04.529689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:04.529689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:04.529689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:04.529689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:04.529689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:04.529689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:04.530926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:04.530926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:04.530926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:04.530926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:04.530926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:04.530926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:04.530926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:04.530926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:04.530926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:04.530926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:04.530926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:04.530926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:04.530926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:04.530926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:04.530926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:04.530926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:04.530926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:04.532108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:04.532108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:04.532108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:04.533999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 384/788] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:04.539922 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65721387-0353-45d8-83d0-231f2852dce6 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:04.542027 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:04.546497 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:04.566431 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11687cd5-21e5-4787-8aa8-a8dc7cd2a925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:04.567524 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11687cd5-21e5-4787-8aa8-a8dc7cd2a925 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:04.569170 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11687cd5-21e5-4787-8aa8-a8dc7cd2a925 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:04.569605 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11687cd5-21e5-4787-8aa8-a8dc7cd2a925 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:04.570673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11687cd5-21e5-4787-8aa8-a8dc7cd2a925 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:04.571858 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11687cd5-21e5-4787-8aa8-a8dc7cd2a925 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:04.575512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11687cd5-21e5-4787-8aa8-a8dc7cd2a925 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:04.596728 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 405/789] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:11:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:04.602173 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11687cd5-21e5-4787-8aa8-a8dc7cd2a925 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:04.747563 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65721387-0353-45d8-83d0-231f2852dce6 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:04.812365 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11687cd5-21e5-4787-8aa8-a8dc7cd2a925 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:04.815167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909158f-7eb3-430b-b288-db3f7b72767e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:04.815861 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909158f-7eb3-430b-b288-db3f7b72767e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:04.816147 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909158f-7eb3-430b-b288-db3f7b72767e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:04.816423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909158f-7eb3-430b-b288-db3f7b72767e None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:04.817069 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909158f-7eb3-430b-b288-db3f7b72767e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:04.817376 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909158f-7eb3-430b-b288-db3f7b72767e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:04.818762 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909158f-7eb3-430b-b288-db3f7b72767e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:04.843710 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 385/790] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:04.857809 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909158f-7eb3-430b-b288-db3f7b72767e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:04.880354 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef5d02-043c-47d3-8bbc-b68d5d14af55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:04.881791 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef5d02-043c-47d3-8bbc-b68d5d14af55 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:04.882076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef5d02-043c-47d3-8bbc-b68d5d14af55 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:04.882351 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef5d02-043c-47d3-8bbc-b68d5d14af55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:04.897358 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef5d02-043c-47d3-8bbc-b68d5d14af55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:04.899327 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef5d02-043c-47d3-8bbc-b68d5d14af55 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:04.914322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-daef5d02-043c-47d3-8bbc-b68d5d14af55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:04.914322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:04.914322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:04.914322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:04.914322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:04.914322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:04.914322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:04.914322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:04.914322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:04.914322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:04.914322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:04.914322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:04.914322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:04.914322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:04.914322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:04.914322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:04.915819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:04.915819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:04.915819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:04.915819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:04.915819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:04.915819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:04.915819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:04.915819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:04.915819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:04.915819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:04.915819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:04.915819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:04.915819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:04.915819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:04.915819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:04.915819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:04.915819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:04.917139 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:04.917139 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:04.917139 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:04.917139 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 406/791] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:04.926569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef5d02-043c-47d3-8bbc-b68d5d14af55 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:04.945178 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:05.013504 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909158f-7eb3-430b-b288-db3f7b72767e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:05.015425 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:05.088182 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b76fb8-e5bb-40ff-9e3c-b298c1898a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:05.088858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b76fb8-e5bb-40ff-9e3c-b298c1898a08 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:05.089141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b76fb8-e5bb-40ff-9e3c-b298c1898a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:05.089416 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b76fb8-e5bb-40ff-9e3c-b298c1898a08 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:05.108999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b76fb8-e5bb-40ff-9e3c-b298c1898a08 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:05.110488 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b76fb8-e5bb-40ff-9e3c-b298c1898a08 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:05.129203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-c7b76fb8-e5bb-40ff-9e3c-b298c1898a08 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:11:05.129203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:05.129203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:05.129203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:05.129203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:05.129203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:05.129203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:05.129203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:05.129203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:05.129203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:05.129203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:05.129203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:05.129203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:11:05.129203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:11:05.129203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:11:05.129203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:11:05.130206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:05.130206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:05.130206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:11:05.130206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:05.130206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:11:05.130206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:05.130206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:05.130206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:05.130206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:05.130206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:05.130206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:05.130206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:05.130206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:05.130206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:05.130206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:05.130206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:05.130206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:05.130206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:05.132449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:05.132449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:05.132449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:11:05.132449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:11:05.132449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:11:05.132449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:11:05.132449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:11:05.132449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:11:05.132449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:11:05.132449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:05.134261 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef5d02-043c-47d3-8bbc-b68d5d14af55 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:05.134413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 386/792] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:11:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:05.140264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b76fb8-e5bb-40ff-9e3c-b298c1898a08 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:05.174503 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842cc7e-e7cf-4839-b08d-e0fed0728cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:05.175119 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842cc7e-e7cf-4839-b08d-e0fed0728cbb None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:05.175400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842cc7e-e7cf-4839-b08d-e0fed0728cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:05.175665 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842cc7e-e7cf-4839-b08d-e0fed0728cbb None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:05.176570 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842cc7e-e7cf-4839-b08d-e0fed0728cbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:05.176874 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842cc7e-e7cf-4839-b08d-e0fed0728cbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:05.178353 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842cc7e-e7cf-4839-b08d-e0fed0728cbb None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:05.197723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 407/793] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:05.199025 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4cacc-74e8-4fae-bfac-ae135eba0c27 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:05.199309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4cacc-74e8-4fae-bfac-ae135eba0c27 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:05.199689 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4cacc-74e8-4fae-bfac-ae135eba0c27 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:05.202656 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 408/794] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:05.203829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4cacc-74e8-4fae-bfac-ae135eba0c27 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:05.283835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b76fb8-e5bb-40ff-9e3c-b298c1898a08 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:05.309811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:05.322161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3a309-3899-48a3-bb7c-eb779d434bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:05.323045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3a309-3899-48a3-bb7c-eb779d434bcd None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:05.323529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3a309-3899-48a3-bb7c-eb779d434bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:05.323894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3a309-3899-48a3-bb7c-eb779d434bcd None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:05.324687 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3a309-3899-48a3-bb7c-eb779d434bcd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:05.325107 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3a309-3899-48a3-bb7c-eb779d434bcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:05.326847 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3a309-3899-48a3-bb7c-eb779d434bcd None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:05.339257 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4cacc-74e8-4fae-bfac-ae135eba0c27 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:05.341168 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:05.350323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 387/795] 10.222.0.81 () {56 vars in 1135 bytes} [Mon Sep 19 11:11:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:05.351872 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f065475-a075-438a-8e6f-2f6e52625498 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:05.352164 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f065475-a075-438a-8e6f-2f6e52625498 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:05.352439 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f065475-a075-438a-8e6f-2f6e52625498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:05.372978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-4f065475-a075-438a-8e6f-2f6e52625498 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:05.374308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4f065475-a075-438a-8e6f-2f6e52625498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:05.379179 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03995d4-e494-41ff-af92-7b094c3cc185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:05.380192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03995d4-e494-41ff-af92-7b094c3cc185 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:05.380673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03995d4-e494-41ff-af92-7b094c3cc185 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:05.381141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03995d4-e494-41ff-af92-7b094c3cc185 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:05.388783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03995d4-e494-41ff-af92-7b094c3cc185 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:05.390428 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03995d4-e494-41ff-af92-7b094c3cc185 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:05.402938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-e03995d4-e494-41ff-af92-7b094c3cc185 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:11:05.402938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:05.402938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:05.402938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:05.402938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:05.402938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:05.402938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:05.402938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:05.402938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:05.402938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:05.402938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:05.402938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:05.402938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:11:05.402938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:11:05.402938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:11:05.402938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:11:05.403890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:05.403890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:05.403890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:11:05.403890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:05.403890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:11:05.403890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:05.403890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:05.403890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:05.403890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:05.403890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:05.403890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:05.403890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:05.403890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:05.403890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:05.403890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:05.403890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:05.403890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:05.403890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:05.404840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:05.404840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:05.404840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:11:05.404840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:11:05.404840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:11:05.404840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:11:05.404840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:11:05.404840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:11:05.404840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:11:05.404840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:05.407343 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 409/796] 10.222.0.81 () {56 vars in 1093 bytes} [Mon Sep 19 11:11:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:05.408437 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03995d4-e494-41ff-af92-7b094c3cc185 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:05.451787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 388/797] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:05.451787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f065475-a075-438a-8e6f-2f6e52625498 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:05.558771 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f065475-a075-438a-8e6f-2f6e52625498 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:05.572909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:05.583502 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03995d4-e494-41ff-af92-7b094c3cc185 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:05.597722 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8ba85-cab0-41db-b5a4-f403928c8de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:05.598664 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8ba85-cab0-41db-b5a4-f403928c8de6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:05.599108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8ba85-cab0-41db-b5a4-f403928c8de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:05.599507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8ba85-cab0-41db-b5a4-f403928c8de6 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:05.600396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc8ba85-cab0-41db-b5a4-f403928c8de6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:05.600829 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc8ba85-cab0-41db-b5a4-f403928c8de6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:05.602464 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc8ba85-cab0-41db-b5a4-f403928c8de6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:05.622710 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 389/798] 10.222.0.81 () {56 vars in 1133 bytes} [Mon Sep 19 11:11:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:05.628116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8ba85-cab0-41db-b5a4-f403928c8de6 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:05.663512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa1ea6-7fbf-4fe9-b870-09254dd4fd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:05.664484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa1ea6-7fbf-4fe9-b870-09254dd4fd19 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:05.664865 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa1ea6-7fbf-4fe9-b870-09254dd4fd19 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:05.665234 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa1ea6-7fbf-4fe9-b870-09254dd4fd19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:05.677670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa1ea6-7fbf-4fe9-b870-09254dd4fd19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:05.680030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa1ea6-7fbf-4fe9-b870-09254dd4fd19 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:05.696006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-b0fa1ea6-7fbf-4fe9-b870-09254dd4fd19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:05.696006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:05.696006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:05.696006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:05.696006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:05.696006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:05.696006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:05.696006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:05.696006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:05.696006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:05.696006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:05.696006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:05.696006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:05.696006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:05.696006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:05.696006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:05.697177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:05.697177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:05.697177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:05.697177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:05.697177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:05.697177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:05.697177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:05.697177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:05.697177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:05.697177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:05.697177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:05.697177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:05.697177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:05.697177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:05.697177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:05.697177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:05.697177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:05.698370 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:05.698370 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:05.698370 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:05.700121 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 410/799] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:05.701678 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050e044-5d4d-4f0a-b258-8a4032efa8b2 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:05.702065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050e044-5d4d-4f0a-b258-8a4032efa8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:05.702444 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050e044-5d4d-4f0a-b258-8a4032efa8b2 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:05.704521 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 411/800] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:05.705303 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050e044-5d4d-4f0a-b258-8a4032efa8b2 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:05.725620 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:05.752016 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8ba85-cab0-41db-b5a4-f403928c8de6 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:05.795021 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85849e-6655-46d6-acf5-0cfddab2a451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:05.795807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85849e-6655-46d6-acf5-0cfddab2a451 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:05.796195 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85849e-6655-46d6-acf5-0cfddab2a451 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:05.796935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85849e-6655-46d6-acf5-0cfddab2a451 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:05.825762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050e044-5d4d-4f0a-b258-8a4032efa8b2 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:05.842894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:05.868460 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ebb42-28b0-4958-a881-d5d46b1d5ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:05.869060 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ebb42-28b0-4958-a881-d5d46b1d5ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:05.869333 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ebb42-28b0-4958-a881-d5d46b1d5ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:05.869597 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ebb42-28b0-4958-a881-d5d46b1d5ad4 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:05.870222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ebb42-28b0-4958-a881-d5d46b1d5ad4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:05.870524 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ebb42-28b0-4958-a881-d5d46b1d5ad4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:05.871897 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ebb42-28b0-4958-a881-d5d46b1d5ad4 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:05.873146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b85849e-6655-46d6-acf5-0cfddab2a451 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:05.874582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b85849e-6655-46d6-acf5-0cfddab2a451 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:05.884327 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 412/801] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:05.885379 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349f071-7018-461d-b999-ce57b5dafba5 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:05.885702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349f071-7018-461d-b999-ce57b5dafba5 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:05.885965 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349f071-7018-461d-b999-ce57b5dafba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:05.904171 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-0349f071-7018-461d-b999-ce57b5dafba5 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:05.905365 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-0349f071-7018-461d-b999-ce57b5dafba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:05.974942 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 413/802] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:05.980617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349f071-7018-461d-b999-ce57b5dafba5 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:05.980811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 390/803] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:11:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:11:05.981708 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85849e-6655-46d6-acf5-0cfddab2a451 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:05.986418 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:06.070144 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:06.113022 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85849e-6655-46d6-acf5-0cfddab2a451 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:06.114829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349f071-7018-461d-b999-ce57b5dafba5 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:06.148139 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725d45a-cf44-447f-a3cb-71d6a2ad7aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:06.149271 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725d45a-cf44-447f-a3cb-71d6a2ad7aab None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:06.149550 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725d45a-cf44-447f-a3cb-71d6a2ad7aab None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:06.149820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725d45a-cf44-447f-a3cb-71d6a2ad7aab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:06.151751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e247e6ac-953d-4f78-b934-993189492ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:06.152400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247e6ac-953d-4f78-b934-993189492ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:06.152679 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247e6ac-953d-4f78-b934-993189492ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:06.152965 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247e6ac-953d-4f78-b934-993189492ad2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:06.157197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725d45a-cf44-447f-a3cb-71d6a2ad7aab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:06.158819 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725d45a-cf44-447f-a3cb-71d6a2ad7aab None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:06.162834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e247e6ac-953d-4f78-b934-993189492ad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:11:06.167670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-e247e6ac-953d-4f78-b934-993189492ad2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:11:06.167670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:06.167670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:06.167670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:06.167670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:06.167670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:06.167670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:06.167670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:06.167670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:06.167670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:06.167670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:06.167670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:06.167670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:11:06.167670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:11:06.167670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:11:06.167670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:11:06.168680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:11:06.168680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:11:06.168680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:11:06.168680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:11:06.168680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:06.168680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:06.168680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:06.168680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:06.168680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:06.168680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:06.168680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:06.168680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:06.168680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:06.168680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:06.168680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:06.168680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:06.169526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:06.169526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:06.169526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:06.169526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:06.169526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:11:06.169526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:11:06.169526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:11:06.169526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:11:06.169526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:11:06.169526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:11:06.169526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:11:06.169526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:06.169526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 414/804] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:11:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:06.169526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e247e6ac-953d-4f78-b934-993189492ad2 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:06.174368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-3725d45a-cf44-447f-a3cb-71d6a2ad7aab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:06.174368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:06.174368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:06.174368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:06.174368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:06.174368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:06.174368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:06.174368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:06.174368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:06.174368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:06.174368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:06.174368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:06.174368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:11:06.174368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:11:06.174368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:11:06.175272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:11:06.175272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:06.175272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:06.175272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:06.175272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:06.175272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:06.175272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:06.175272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:06.175272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:06.175272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:06.175272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:06.175272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:06.175272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:06.175272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:06.175272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:06.175272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:06.175272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:06.176220 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:11:06.176220 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:11:06.176220 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:11:06.176220 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:11:06.176220 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:11:06.176220 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:11:06.176220 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:06.176220 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:06.177070 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 391/805] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:11:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:06.178189 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9e9a3-abde-418c-a189-8b9e61198ded None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:06.178469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9e9a3-abde-418c-a189-8b9e61198ded None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:06.178747 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9e9a3-abde-418c-a189-8b9e61198ded None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:06.180727 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 392/806] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:06.181422 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9e9a3-abde-418c-a189-8b9e61198ded None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:06.241992 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:06.282124 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e247e6ac-953d-4f78-b934-993189492ad2 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:06.288642 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9e9a3-abde-418c-a189-8b9e61198ded None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:06.317126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:06.317293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae7122-325a-404a-a3f9-7301527311b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:06.317825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae7122-325a-404a-a3f9-7301527311b3 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:06.318156 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae7122-325a-404a-a3f9-7301527311b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:06.318546 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae7122-325a-404a-a3f9-7301527311b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:06.323624 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa6555-bcad-415f-89c2-2c37d32377df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:06.324252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa6555-bcad-415f-89c2-2c37d32377df None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:06.324532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa6555-bcad-415f-89c2-2c37d32377df None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:06.324803 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa6555-bcad-415f-89c2-2c37d32377df None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:06.325458 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa6555-bcad-415f-89c2-2c37d32377df None admin] RBAC: Adding query filter params (name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:06.325760 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa6555-bcad-415f-89c2-2c37d32377df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:06.327124 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa6555-bcad-415f-89c2-2c37d32377df None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:06.329313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae7122-325a-404a-a3f9-7301527311b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:06.330737 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae7122-325a-404a-a3f9-7301527311b3 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:06.335692 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 393/807] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:11:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:06.337063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa6555-bcad-415f-89c2-2c37d32377df None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:06.446912 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa6555-bcad-415f-89c2-2c37d32377df None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:06.460611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-f5ae7122-325a-404a-a3f9-7301527311b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:06.460611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:06.460611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:06.460611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:06.460611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:06.460611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:06.460611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:06.460611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:06.460611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:06.460611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:06.460611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:06.460611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:06.460611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:06.460611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:06.460611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:06.460611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:06.462214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:06.462214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:06.462214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:06.462214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:06.462214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:06.462214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:06.462214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:06.462214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:06.462214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:06.462214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:06.462214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:06.462214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:06.462214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:06.462214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:06.462214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:06.462214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:06.462214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:06.463125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:06.463125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:06.463125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:06.463125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 415/808] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:06.463618 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db650002-11ed-4d3e-8c52-aabffda883f2 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:06.463618 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db650002-11ed-4d3e-8c52-aabffda883f2 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:06.463930 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db650002-11ed-4d3e-8c52-aabffda883f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:06.481107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-db650002-11ed-4d3e-8c52-aabffda883f2 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:06.482310 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-db650002-11ed-4d3e-8c52-aabffda883f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:06.485473 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996577f3-c8c4-45b2-ada2-6dd07e83f429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:06.486068 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996577f3-c8c4-45b2-ada2-6dd07e83f429 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:06.486349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996577f3-c8c4-45b2-ada2-6dd07e83f429 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:06.486620 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996577f3-c8c4-45b2-ada2-6dd07e83f429 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:06.487309 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996577f3-c8c4-45b2-ada2-6dd07e83f429 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:06.487616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996577f3-c8c4-45b2-ada2-6dd07e83f429 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:06.489064 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996577f3-c8c4-45b2-ada2-6dd07e83f429 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:06.500444 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 394/809] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:11:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:06.501236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996577f3-c8c4-45b2-ada2-6dd07e83f429 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:06.549133 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 416/810] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:06.550178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db650002-11ed-4d3e-8c52-aabffda883f2 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:06.552318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:06.600422 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:06.602917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996577f3-c8c4-45b2-ada2-6dd07e83f429 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:06.641402 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b6e04b-fe61-4a5d-913f-17a9bc63e362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:06.642629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6e04b-fe61-4a5d-913f-17a9bc63e362 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:06.643475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6e04b-fe61-4a5d-913f-17a9bc63e362 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:06.644007 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6e04b-fe61-4a5d-913f-17a9bc63e362 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:06.645027 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b6e04b-fe61-4a5d-913f-17a9bc63e362 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:06.647066 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b6e04b-fe61-4a5d-913f-17a9bc63e362 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:06.658399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-d2b6e04b-fe61-4a5d-913f-17a9bc63e362 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:11:06.658399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:06.658399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:06.658399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:06.658399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:06.658399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:06.658399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:06.658399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:06.658399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:11:06.658399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:06.658399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:06.658399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:06.658399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:06.658399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:06.658399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:06.658399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:06.658399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:06.659409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:06.659409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:06.659409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:06.659409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:06.659409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:06.659409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:06.659409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:06.659409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:06.659409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:06.659409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:06.659409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:06.659409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:06.659409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:06.659409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:06.659409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:06.659409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:06.660333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:06.660333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:06.660333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:06.660333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:06.660333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:06.660333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:06.660333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:06.660333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:06.660333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:06.660333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:06.660333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:06.660333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:06.660333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 11:11:06.660333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:06.660333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 395/811] 10.222.0.81 () {56 vars in 1129 bytes} [Mon Sep 19 11:11:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:06.661125 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b6e04b-fe61-4a5d-913f-17a9bc63e362 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:06.665592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db650002-11ed-4d3e-8c52-aabffda883f2 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:06.705016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4cb28-3bf9-479d-a418-a639547e5cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:06.705661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4cb28-3bf9-479d-a418-a639547e5cee None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:06.706017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4cb28-3bf9-479d-a418-a639547e5cee None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:06.706532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4cb28-3bf9-479d-a418-a639547e5cee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:06.716738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4cb28-3bf9-479d-a418-a639547e5cee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:06.718167 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4cb28-3bf9-479d-a418-a639547e5cee None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:06.733697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-feb4cb28-3bf9-479d-a418-a639547e5cee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:06.733697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:06.733697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:06.733697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:06.733697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:06.733697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:06.733697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:06.733697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:06.733697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:06.733697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:06.733697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:06.733697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:06.733697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:06.733697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:06.733697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:06.733697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:06.734668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:06.734668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:06.734668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:06.734668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:06.734668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:06.734668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:06.734668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:06.734668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:06.734668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:06.734668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:06.734668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:06.734668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:06.734668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:06.734668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:06.734668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:06.734668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:06.734668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:06.735584 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:06.735584 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:06.735584 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:06.735584 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 417/812] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:06.735584 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4cb28-3bf9-479d-a418-a639547e5cee None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:06.773648 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b6e04b-fe61-4a5d-913f-17a9bc63e362 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:06.790635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:06.809527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaae82d-5678-4d9c-8933-06a007bcde50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:06.810135 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaae82d-5678-4d9c-8933-06a007bcde50 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:06.810416 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaae82d-5678-4d9c-8933-06a007bcde50 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:06.810693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaae82d-5678-4d9c-8933-06a007bcde50 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:06.811338 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaae82d-5678-4d9c-8933-06a007bcde50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:06.811640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaae82d-5678-4d9c-8933-06a007bcde50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:06.813061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaae82d-5678-4d9c-8933-06a007bcde50 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:06.824849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 396/813] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:06.825649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaae82d-5678-4d9c-8933-06a007bcde50 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:06.844942 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:06.849799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4cb28-3bf9-479d-a418-a639547e5cee None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:06.884062 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cac145-dfab-4bbe-a239-2c780ea65f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:06.884730 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cac145-dfab-4bbe-a239-2c780ea65f70 None admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:06.885003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cac145-dfab-4bbe-a239-2c780ea65f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:06.885274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cac145-dfab-4bbe-a239-2c780ea65f70 None admin] PATH_INFO: `/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/1c8a9732807d4921a41a6a2d5019e2f5/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:06.887185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cac145-dfab-4bbe-a239-2c780ea65f70 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed9a0b626e3e4b93b391158e981ec586, user_id=1c8a9732807d4921a41a6a2d5019e2f5, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:06.888623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cac145-dfab-4bbe-a239-2c780ea65f70 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:06.900985 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 418/814] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:11:06 2022] PUT /identity/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/1c8a9732807d4921a41a6a2d5019e2f5/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:11:06.901600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cac145-dfab-4bbe-a239-2c780ea65f70 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:06.927117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaae82d-5678-4d9c-8933-06a007bcde50 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:06.968504 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655508f-1e59-40b3-910b-2afd8243f230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:06.969154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655508f-1e59-40b3-910b-2afd8243f230 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:06.969434 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655508f-1e59-40b3-910b-2afd8243f230 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:06.969701 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655508f-1e59-40b3-910b-2afd8243f230 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:06.970334 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655508f-1e59-40b3-910b-2afd8243f230 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:06.970630 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655508f-1e59-40b3-910b-2afd8243f230 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:06.972078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655508f-1e59-40b3-910b-2afd8243f230 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:06.977178 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 397/815] 10.222.0.81 () {56 vars in 1124 bytes} [Mon Sep 19 11:11:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:06.978043 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655508f-1e59-40b3-910b-2afd8243f230 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:07.027127 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:07.035377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cac145-dfab-4bbe-a239-2c780ea65f70 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:07.077435 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fa2cf-2d77-4199-b17f-24a695489562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:07.078609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fa2cf-2d77-4199-b17f-24a695489562 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:07.079075 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fa2cf-2d77-4199-b17f-24a695489562 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:07.079538 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fa2cf-2d77-4199-b17f-24a695489562 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:07.080425 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fa2cf-2d77-4199-b17f-24a695489562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:07.080941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fa2cf-2d77-4199-b17f-24a695489562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:07.082462 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fa2cf-2d77-4199-b17f-24a695489562 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:07.093613 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655508f-1e59-40b3-910b-2afd8243f230 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:07.097226 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 419/816] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:07.097875 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fa2cf-2d77-4199-b17f-24a695489562 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:07.120035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:07.128782 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ae335-4da9-4c2d-b2da-443e0d408c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:07.129427 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ae335-4da9-4c2d-b2da-443e0d408c68 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:07.129725 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ae335-4da9-4c2d-b2da-443e0d408c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:07.130044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ae335-4da9-4c2d-b2da-443e0d408c68 None admin] PATH_INFO: `/v3/services/network` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:07.130613 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ae335-4da9-4c2d-b2da-443e0d408c68 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:07.132036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ae335-4da9-4c2d-b2da-443e0d408c68 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:07.141020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-f76ae335-4da9-4c2d-b2da-443e0d408c68 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:11:07.141020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:07.141020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:07.141020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:07.141020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:07.141020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:07.141020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:07.141020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:07.141020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:11:07.141020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:07.141020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:07.141020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:07.141020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:07.141020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:07.141020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:07.141020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:07.141020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:07.142018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:07.142018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:07.142018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:07.142018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:07.142018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:07.142018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:07.142018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:07.142018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:07.142018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:07.142018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:07.142018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:07.142018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:07.142018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:07.142018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:07.142018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:07.142018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:07.142892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:07.142892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:07.142892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:07.142892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:07.142892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:07.142892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:07.142892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:07.142892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:07.142892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:07.142892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:07.142892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:07.142892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:07.142892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 11:11:07.142892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:07.142892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 398/817] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:11:06 2022] GET /identity/v3/services/network => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:07.143707 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ae335-4da9-4c2d-b2da-443e0d408c68 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:07.209505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fa2cf-2d77-4199-b17f-24a695489562 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:07.244432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289c0cf-8a91-4123-bbcc-24e26599a357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:07.244980 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289c0cf-8a91-4123-bbcc-24e26599a357 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:07.245249 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289c0cf-8a91-4123-bbcc-24e26599a357 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:07.245515 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289c0cf-8a91-4123-bbcc-24e26599a357 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:07.252404 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289c0cf-8a91-4123-bbcc-24e26599a357 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:07.253814 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289c0cf-8a91-4123-bbcc-24e26599a357 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:07.253985 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ae335-4da9-4c2d-b2da-443e0d408c68 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:07.257423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:07.265673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-3289c0cf-8a91-4123-bbcc-24e26599a357 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:07.265673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:07.265673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:07.265673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:07.265673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:07.265673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:07.265673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:07.265673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:07.265673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:07.265673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:07.265673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:07.265673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:07.265673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:11:07.265673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:11:07.265673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:11:07.266600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:11:07.266600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:07.266600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:07.266600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:07.266600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:07.266600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:07.266600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:07.266600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:07.266600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:07.266600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:07.266600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:07.266600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:07.266600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:07.266600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:07.266600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:07.266600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:07.266600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:07.267537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:11:07.267537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:11:07.267537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:11:07.267537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:11:07.267537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:11:07.267537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:11:07.267537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:07.267537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:07.272115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 420/818] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:11:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:07.273034 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289c0cf-8a91-4123-bbcc-24e26599a357 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:07.294008 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab338281-8d3c-43ac-b57b-ddcc96596c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:07.294613 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab338281-8d3c-43ac-b57b-ddcc96596c18 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:07.294892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab338281-8d3c-43ac-b57b-ddcc96596c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:07.295167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab338281-8d3c-43ac-b57b-ddcc96596c18 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:07.295804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab338281-8d3c-43ac-b57b-ddcc96596c18 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:07.296408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab338281-8d3c-43ac-b57b-ddcc96596c18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:07.297471 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab338281-8d3c-43ac-b57b-ddcc96596c18 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:07.307145 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 399/819] 10.222.0.81 () {56 vars in 1124 bytes} [Mon Sep 19 11:11:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:07.307145 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab338281-8d3c-43ac-b57b-ddcc96596c18 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:07.361349 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:07.387433 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289c0cf-8a91-4123-bbcc-24e26599a357 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:07.415710 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab338281-8d3c-43ac-b57b-ddcc96596c18 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:07.426953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbab2f-14c3-4def-b8bc-0eca9e0d1c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:07.427738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbab2f-14c3-4def-b8bc-0eca9e0d1c52 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:07.428052 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbab2f-14c3-4def-b8bc-0eca9e0d1c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:07.428323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbab2f-14c3-4def-b8bc-0eca9e0d1c52 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:07.436786 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbab2f-14c3-4def-b8bc-0eca9e0d1c52 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:07.438212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbab2f-14c3-4def-b8bc-0eca9e0d1c52 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:07.449962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-65dbab2f-14c3-4def-b8bc-0eca9e0d1c52 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:11:07.449962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:07.449962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:07.449962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:07.449962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:07.449962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:07.449962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:07.449962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:07.449962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:07.449962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:07.449962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:07.449962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:07.449962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:11:07.449962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:11:07.449962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:11:07.449962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:11:07.450911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:07.450911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:07.450911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:11:07.450911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:07.450911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:11:07.450911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:07.450911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:07.450911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:07.450911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:07.450911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:07.450911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:07.450911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:07.450911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:07.450911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:07.450911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:07.450911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:07.450911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:07.450911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:07.451880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:07.451880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:07.451880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:11:07.451880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:11:07.451880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:11:07.451880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:11:07.451880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:11:07.451880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:11:07.451880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:11:07.451880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:07.452422 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191bbd3-6a19-4171-8c5f-fef3a9b8ed0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:07.452422 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191bbd3-6a19-4171-8c5f-fef3a9b8ed0e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:07.452422 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191bbd3-6a19-4171-8c5f-fef3a9b8ed0e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:07.452422 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191bbd3-6a19-4171-8c5f-fef3a9b8ed0e None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:07.452687 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191bbd3-6a19-4171-8c5f-fef3a9b8ed0e None admin] RBAC: Adding query filter params (name=network) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:07.452687 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191bbd3-6a19-4171-8c5f-fef3a9b8ed0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:07.453799 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191bbd3-6a19-4171-8c5f-fef3a9b8ed0e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:07.456258 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 421/820] 10.222.0.81 () {56 vars in 1117 bytes} [Mon Sep 19 11:11:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:07.456790 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbab2f-14c3-4def-b8bc-0eca9e0d1c52 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:07.463318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 400/821] 10.222.0.81 () {56 vars in 1112 bytes} [Mon Sep 19 11:11:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:07.464063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191bbd3-6a19-4171-8c5f-fef3a9b8ed0e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:07.497234 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:07.559913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbab2f-14c3-4def-b8bc-0eca9e0d1c52 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:07.568653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191bbd3-6a19-4171-8c5f-fef3a9b8ed0e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:07.603042 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1888c0-e7ff-47bd-8306-f467ac17b217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:07.603646 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1888c0-e7ff-47bd-8306-f467ac17b217 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:07.603957 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1888c0-e7ff-47bd-8306-f467ac17b217 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:07.604289 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1888c0-e7ff-47bd-8306-f467ac17b217 None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:07.604920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1888c0-e7ff-47bd-8306-f467ac17b217 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:07.605228 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1888c0-e7ff-47bd-8306-f467ac17b217 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:07.605434 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c68e0-c756-46c8-b948-59073bdb30e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:07.606038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c68e0-c756-46c8-b948-59073bdb30e6 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:07.606319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c68e0-c756-46c8-b948-59073bdb30e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:07.606592 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c68e0-c756-46c8-b948-59073bdb30e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:07.606686 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1888c0-e7ff-47bd-8306-f467ac17b217 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:07.607254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c68e0-c756-46c8-b948-59073bdb30e6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:07.608638 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c68e0-c756-46c8-b948-59073bdb30e6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:07.614788 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:07.627006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 422/822] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:11:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:07.628051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1888c0-e7ff-47bd-8306-f467ac17b217 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:07.634202 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 401/823] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:11:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:11:07.635255 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c68e0-c756-46c8-b948-59073bdb30e6 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:07.756514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1888c0-e7ff-47bd-8306-f467ac17b217 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:07.780610 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:07.804257 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c68e0-c756-46c8-b948-59073bdb30e6 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:07.814498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90d8b2-0b78-4195-9623-11b15dce571c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:07.815425 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90d8b2-0b78-4195-9623-11b15dce571c None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:07.815887 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90d8b2-0b78-4195-9623-11b15dce571c None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:07.816696 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90d8b2-0b78-4195-9623-11b15dce571c None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:07.817600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90d8b2-0b78-4195-9623-11b15dce571c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:07.818390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90d8b2-0b78-4195-9623-11b15dce571c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:07.820670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90d8b2-0b78-4195-9623-11b15dce571c None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:07.846055 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ae03a-005b-430c-8851-6834e8a2e586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:07.846373 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 423/824] 10.222.0.81 () {56 vars in 1149 bytes} [Mon Sep 19 11:11:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:07.846880 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ae03a-005b-430c-8851-6834e8a2e586 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:07.847256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ae03a-005b-430c-8851-6834e8a2e586 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:07.847625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ae03a-005b-430c-8851-6834e8a2e586 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:07.847894 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90d8b2-0b78-4195-9623-11b15dce571c None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:07.849242 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ae03a-005b-430c-8851-6834e8a2e586 None admin] RBAC: Adding query filter params (type=network) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:07.849242 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ae03a-005b-430c-8851-6834e8a2e586 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:07.850718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ae03a-005b-430c-8851-6834e8a2e586 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:07.856046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 402/825] 10.222.0.81 () {56 vars in 1112 bytes} [Mon Sep 19 11:11:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:07.856952 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ae03a-005b-430c-8851-6834e8a2e586 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:07.925225 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:07.971741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ae03a-005b-430c-8851-6834e8a2e586 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:07.988185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90d8b2-0b78-4195-9623-11b15dce571c None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:08.007067 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a4fe3-2f55-49fa-b956-f34cff3f7255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:08.007621 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a4fe3-2f55-49fa-b956-f34cff3f7255 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:08.008446 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a4fe3-2f55-49fa-b956-f34cff3f7255 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:08.008722 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a4fe3-2f55-49fa-b956-f34cff3f7255 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:08.019531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a4fe3-2f55-49fa-b956-f34cff3f7255 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:08.021018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a4fe3-2f55-49fa-b956-f34cff3f7255 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:08.029769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:08.035668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636ab70-cd11-41b2-a371-89c4bb67e956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:08.036355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-2d0a4fe3-2f55-49fa-b956-f34cff3f7255 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:08.036355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:08.036355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:08.036355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:08.036355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:08.036355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:08.036355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:08.036355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:08.036355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:08.036355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:08.036355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:08.036355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:08.036355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:08.036355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:08.036355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:08.036355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:08.037503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:08.037503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:08.037503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:08.037503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:08.037503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:08.037503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:08.037503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:08.037503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:08.037503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:08.037503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:08.037503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:08.037503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:08.037503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:08.037503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:08.037503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:08.037503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:08.037503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:08.038605 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:08.038605 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:08.038605 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:08.040021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636ab70-cd11-41b2-a371-89c4bb67e956 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:08.040456 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636ab70-cd11-41b2-a371-89c4bb67e956 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:08.041265 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636ab70-cd11-41b2-a371-89c4bb67e956 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:08.042131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636ab70-cd11-41b2-a371-89c4bb67e956 None admin] RBAC: Adding query filter params (role.id=753f8b35d5f0458d8f62d0ad6e28033f, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=5abd70a18ccd407bafe3b61c8870986e) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:08.042585 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636ab70-cd11-41b2-a371-89c4bb67e956 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:08.044152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636ab70-cd11-41b2-a371-89c4bb67e956 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:08.045691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 403/826] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:08.046621 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a4fe3-2f55-49fa-b956-f34cff3f7255 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:08.059225 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 424/827] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:11:07 2022] GET /identity/v3/role_assignments?user.id=5abd70a18ccd407bafe3b61c8870986e&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=753f8b35d5f0458d8f62d0ad6e28033f => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:08.059225 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636ab70-cd11-41b2-a371-89c4bb67e956 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:08.233278 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636ab70-cd11-41b2-a371-89c4bb67e956 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:08.265137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:08.272429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a4fe3-2f55-49fa-b956-f34cff3f7255 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:08.287071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11131970-6411-455e-85fe-def62a60cb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:08.287676 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11131970-6411-455e-85fe-def62a60cb74 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:08.288004 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11131970-6411-455e-85fe-def62a60cb74 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:08.288371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11131970-6411-455e-85fe-def62a60cb74 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:08.289015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11131970-6411-455e-85fe-def62a60cb74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:08.289442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11131970-6411-455e-85fe-def62a60cb74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:08.290799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11131970-6411-455e-85fe-def62a60cb74 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:08.303749 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 425/828] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:08.310194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11131970-6411-455e-85fe-def62a60cb74 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:08.313349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e36383-fa4f-40de-bc77-255e58d2f329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:08.313964 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e36383-fa4f-40de-bc77-255e58d2f329 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:08.314297 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e36383-fa4f-40de-bc77-255e58d2f329 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:08.314579 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e36383-fa4f-40de-bc77-255e58d2f329 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:08.315261 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e36383-fa4f-40de-bc77-255e58d2f329 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:08.316665 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e36383-fa4f-40de-bc77-255e58d2f329 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:08.347951 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 404/829] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:11:08 2022] POST /identity/v3/endpoints => generated 337 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:11:08.434654 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11131970-6411-455e-85fe-def62a60cb74 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:08.483692 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456132f9-c4a0-4604-b287-e322c74d2a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:08.484436 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456132f9-c4a0-4604-b287-e322c74d2a7f None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:08.484711 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456132f9-c4a0-4604-b287-e322c74d2a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:08.484978 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456132f9-c4a0-4604-b287-e322c74d2a7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:08.492967 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456132f9-c4a0-4604-b287-e322c74d2a7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:08.494575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456132f9-c4a0-4604-b287-e322c74d2a7f None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:08.507377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-456132f9-c4a0-4604-b287-e322c74d2a7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:08.507377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:08.507377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:08.507377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:08.507377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:08.507377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:08.507377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:08.507377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:08.507377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:08.507377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:08.507377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:08.507377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:08.507377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:11:08.507377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:11:08.507377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:11:08.508497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:11:08.508497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:08.508497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:08.508497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:08.508497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:08.508497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:08.508497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:08.508497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:08.508497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:08.508497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:08.508497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:08.508497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:08.508497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:08.508497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:08.508497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:08.508497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:08.508497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:08.509442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:11:08.509442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:11:08.509442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:11:08.509442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:11:08.509442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:11:08.509442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:11:08.509442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:08.509442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:08.512083 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 426/830] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:11:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:08.515232 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e36383-fa4f-40de-bc77-255e58d2f329 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:08.584358 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:08.632587 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e36383-fa4f-40de-bc77-255e58d2f329 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:08.671744 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd657e-01a5-4039-a7dd-8968aae507e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:08.672444 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd657e-01a5-4039-a7dd-8968aae507e6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:08.672686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd657e-01a5-4039-a7dd-8968aae507e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:08.672961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd657e-01a5-4039-a7dd-8968aae507e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:08.673671 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd657e-01a5-4039-a7dd-8968aae507e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:08.673978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd657e-01a5-4039-a7dd-8968aae507e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:08.675414 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd657e-01a5-4039-a7dd-8968aae507e6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:08.692036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 405/831] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:11:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:08.694969 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456132f9-c4a0-4604-b287-e322c74d2a7f None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:08.731779 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:08.805851 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456132f9-c4a0-4604-b287-e322c74d2a7f None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:08.841728 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9ae84-ed3b-4de7-a695-1f330de21da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:08.842485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9ae84-ed3b-4de7-a695-1f330de21da6 None admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:08.842768 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9ae84-ed3b-4de7-a695-1f330de21da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:08.843040 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9ae84-ed3b-4de7-a695-1f330de21da6 None admin] PATH_INFO: `/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/06d279b488d148428b7a3c35a9786ffe/roles/af9d39307d714527b14b2045ab466e7a` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:08.858671 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9ae84-ed3b-4de7-a695-1f330de21da6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed9a0b626e3e4b93b391158e981ec586, user_id=06d279b488d148428b7a3c35a9786ffe, role_id=af9d39307d714527b14b2045ab466e7a)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:08.860194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9ae84-ed3b-4de7-a695-1f330de21da6 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:08.879476 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 427/832] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:11:08 2022] PUT /identity/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/06d279b488d148428b7a3c35a9786ffe/roles/af9d39307d714527b14b2045ab466e7a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:11:09.124233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f51ef-dd4a-4085-880a-6d6940e1e4cd None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:09.124619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f51ef-dd4a-4085-880a-6d6940e1e4cd None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:09.124830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f51ef-dd4a-4085-880a-6d6940e1e4cd None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:09.126255 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 406/833] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:09.131741 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2231d2-376d-431b-9bf2-9dcdbc44af4c None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:09.132280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2231d2-376d-431b-9bf2-9dcdbc44af4c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:09.132749 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2231d2-376d-431b-9bf2-9dcdbc44af4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:09.152480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-ed2231d2-376d-431b-9bf2-9dcdbc44af4c None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:09.153994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ed2231d2-376d-431b-9bf2-9dcdbc44af4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:09.224908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 428/834] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:09.506507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f51ef-dd4a-4085-880a-6d6940e1e4cd None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:09.618117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f51ef-dd4a-4085-880a-6d6940e1e4cd None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:09.636216 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:09.658206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb161321-daa8-4bf3-b3ce-17b60a46eb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:09.658851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb161321-daa8-4bf3-b3ce-17b60a46eb4a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:09.659132 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb161321-daa8-4bf3-b3ce-17b60a46eb4a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:09.659406 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb161321-daa8-4bf3-b3ce-17b60a46eb4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:09.670992 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb161321-daa8-4bf3-b3ce-17b60a46eb4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:11:09.676028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-fb161321-daa8-4bf3-b3ce-17b60a46eb4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:11:09.676028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:09.676028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:09.676028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:09.676028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:09.676028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:09.676028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:09.676028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:09.676028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:09.676028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:09.676028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:09.676028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:09.676028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:11:09.676028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:11:09.676028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:11:09.676028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:11:09.676989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:11:09.676989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:11:09.676989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:11:09.676989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:11:09.676989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:09.676989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:09.676989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:09.676989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:09.676989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:09.676989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:09.676989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:09.676989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:09.676989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:09.676989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:09.676989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:09.676989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:09.677824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:09.677824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:09.677824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:09.677824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:09.677824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:11:09.677824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:11:09.677824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:11:09.677824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:11:09.677824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:11:09.677824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:11:09.677824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:11:09.677824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:09.677824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 407/835] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:11:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:09.684668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2231d2-376d-431b-9bf2-9dcdbc44af4c None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:09.700056 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:09.795088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2231d2-376d-431b-9bf2-9dcdbc44af4c None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:09.833580 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43eaa5-b1fd-4707-9790-a68484ef395b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:09.834203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43eaa5-b1fd-4707-9790-a68484ef395b None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:09.834481 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43eaa5-b1fd-4707-9790-a68484ef395b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:09.834755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43eaa5-b1fd-4707-9790-a68484ef395b None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:09.835429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43eaa5-b1fd-4707-9790-a68484ef395b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:09.835733 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43eaa5-b1fd-4707-9790-a68484ef395b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:09.837681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43eaa5-b1fd-4707-9790-a68484ef395b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:09.846926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 429/836] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:11:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:09.853034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb161321-daa8-4bf3-b3ce-17b60a46eb4a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:09.966899 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb161321-daa8-4bf3-b3ce-17b60a46eb4a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:10.003274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd49af-5458-4448-97ca-3a6a7292449e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:10.003936 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd49af-5458-4448-97ca-3a6a7292449e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:10.004218 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd49af-5458-4448-97ca-3a6a7292449e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:10.004490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd49af-5458-4448-97ca-3a6a7292449e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:10.007914 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3aa64a-bd80-468b-90e0-ae368bd0a2c9 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:10.008691 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3aa64a-bd80-468b-90e0-ae368bd0a2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:10.008967 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3aa64a-bd80-468b-90e0-ae368bd0a2c9 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:10.010533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 430/837] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:10.015446 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b505b9e-e7d4-45a4-a180-40fe4826344a None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:10.015729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b505b9e-e7d4-45a4-a180-40fe4826344a None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:10.015849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd49af-5458-4448-97ca-3a6a7292449e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:10.016033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b505b9e-e7d4-45a4-a180-40fe4826344a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:10.017215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd49af-5458-4448-97ca-3a6a7292449e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:10.032353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-f6bd49af-5458-4448-97ca-3a6a7292449e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:10.032353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:10.032353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:10.032353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:10.032353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:10.032353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:10.032353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:10.032353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:10.032353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:10.032353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:10.032353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:10.032353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:10.032353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:10.032353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:10.032353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:10.032353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:10.033288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:10.033288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:10.033288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:10.033288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:10.033288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:10.033288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:10.033288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:10.033288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:10.033288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:10.033288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:10.033288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:10.033288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:10.033288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:10.033288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:10.033288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:10.033288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:10.033288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:10.034175 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:10.034175 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:10.034175 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:10.034175 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 408/838] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:10.034398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-5b505b9e-e7d4-45a4-a180-40fe4826344a None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:10.034995 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5b505b9e-e7d4-45a4-a180-40fe4826344a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:10.043303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd49af-5458-4448-97ca-3a6a7292449e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:10.076189 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:10.110941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 431/839] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:10.158857 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd49af-5458-4448-97ca-3a6a7292449e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:10.193663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c44f1-2556-493f-9db1-0e9a1afc9c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:10.194323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c44f1-2556-493f-9db1-0e9a1afc9c72 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:10.194603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c44f1-2556-493f-9db1-0e9a1afc9c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:10.194872 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c44f1-2556-493f-9db1-0e9a1afc9c72 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:10.196049 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c44f1-2556-493f-9db1-0e9a1afc9c72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:10.196350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c44f1-2556-493f-9db1-0e9a1afc9c72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:10.197726 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c44f1-2556-493f-9db1-0e9a1afc9c72 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:10.210365 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 409/840] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:10.215964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b505b9e-e7d4-45a4-a180-40fe4826344a None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:10.220507 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:10.326188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b505b9e-e7d4-45a4-a180-40fe4826344a None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:10.361728 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049bb91-9375-41c3-b5ad-2c00a5c1596c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:10.362351 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049bb91-9375-41c3-b5ad-2c00a5c1596c None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:10.362622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049bb91-9375-41c3-b5ad-2c00a5c1596c None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:10.362885 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049bb91-9375-41c3-b5ad-2c00a5c1596c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:10.370698 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049bb91-9375-41c3-b5ad-2c00a5c1596c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:10.372212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049bb91-9375-41c3-b5ad-2c00a5c1596c None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:10.380999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c44f1-2556-493f-9db1-0e9a1afc9c72 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:10.384356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-9049bb91-9375-41c3-b5ad-2c00a5c1596c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:11:10.384356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:10.384356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:10.384356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:10.384356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:10.384356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:10.384356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:10.384356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:10.384356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:10.384356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:10.384356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:10.384356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:10.384356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:11:10.384356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:11:10.384356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:11:10.384356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:11:10.385380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:10.385380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:10.385380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:11:10.385380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:10.385380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:11:10.385380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:10.385380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:10.385380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:10.385380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:10.385380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:10.385380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:10.385380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:10.385380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:10.385380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:10.385380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:10.385380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:10.385380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:10.385380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:10.386392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:10.386392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:10.386392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:11:10.386392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:11:10.386392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:11:10.386392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:11:10.386392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:11:10.386392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:11:10.386392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 11:11:10.386392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:10.386392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 432/841] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:11:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:10.392170 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049bb91-9375-41c3-b5ad-2c00a5c1596c None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:10.469477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:10.503295 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:10.518869 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049bb91-9375-41c3-b5ad-2c00a5c1596c None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:10.519579 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c44f1-2556-493f-9db1-0e9a1afc9c72 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:10.560958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efd634-da2a-41e3-94d3-440a6bf3c606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:10.561569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efd634-da2a-41e3-94d3-440a6bf3c606 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:10.561946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efd634-da2a-41e3-94d3-440a6bf3c606 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:10.562216 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efd634-da2a-41e3-94d3-440a6bf3c606 None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:10.563178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5efd634-da2a-41e3-94d3-440a6bf3c606 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:10.563483 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5efd634-da2a-41e3-94d3-440a6bf3c606 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:10.564915 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37c4ac-4f00-4697-90a3-fea976452f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:10.565142 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5efd634-da2a-41e3-94d3-440a6bf3c606 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:10.565668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37c4ac-4f00-4697-90a3-fea976452f6f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:10.566017 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37c4ac-4f00-4697-90a3-fea976452f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:10.566359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37c4ac-4f00-4697-90a3-fea976452f6f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:10.567016 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37c4ac-4f00-4697-90a3-fea976452f6f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:10.568493 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37c4ac-4f00-4697-90a3-fea976452f6f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:10.577569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-9c37c4ac-4f00-4697-90a3-fea976452f6f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:11:10.577569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:10.577569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:10.577569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:10.577569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:10.577569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:10.577569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:10.577569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:10.577569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:11:10.577569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:10.577569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:10.577569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:10.577569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:10.577569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:10.577569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:10.577569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:10.577569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:10.578842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:10.578842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:10.578842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:10.578842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:10.578842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:10.578842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:10.578842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:10.578842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:10.578842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:10.578842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:10.578842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:10.578842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:10.578842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:10.578842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:10.578842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:10.578842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:10.579685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:10.579685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:10.579685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:10.579685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:10.579685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:10.579685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:10.579685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:10.579685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:10.579685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:10.579685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:10.579685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:10.579685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:10.579685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:11:10.579685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:10.579685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 410/842] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:11:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:10.586910 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37c4ac-4f00-4697-90a3-fea976452f6f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:10.589919 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 433/843] 10.222.0.81 () {56 vars in 1135 bytes} [Mon Sep 19 11:11:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:10.596055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efd634-da2a-41e3-94d3-440a6bf3c606 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:10.714893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37c4ac-4f00-4697-90a3-fea976452f6f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:10.735917 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efd634-da2a-41e3-94d3-440a6bf3c606 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:10.755399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f475bf-6cc2-4e65-b576-55150d03b7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:10.756381 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f475bf-6cc2-4e65-b576-55150d03b7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:10.756885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f475bf-6cc2-4e65-b576-55150d03b7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:10.757439 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f475bf-6cc2-4e65-b576-55150d03b7b9 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:10.758425 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f475bf-6cc2-4e65-b576-55150d03b7b9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:10.758926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f475bf-6cc2-4e65-b576-55150d03b7b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:10.760407 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:10.761419 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f475bf-6cc2-4e65-b576-55150d03b7b9 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:10.766853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 411/844] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:11:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:10.773082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f475bf-6cc2-4e65-b576-55150d03b7b9 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:10.778509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ead224-6c94-400a-8187-f71c8bfe8b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:10.778759 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ead224-6c94-400a-8187-f71c8bfe8b68 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:10.779033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ead224-6c94-400a-8187-f71c8bfe8b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:10.779301 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ead224-6c94-400a-8187-f71c8bfe8b68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:10.795804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:10.851569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ead224-6c94-400a-8187-f71c8bfe8b68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:10.853037 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ead224-6c94-400a-8187-f71c8bfe8b68 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:10.876942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f475bf-6cc2-4e65-b576-55150d03b7b9 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:10.910816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f1db6-67e3-414d-b2ee-c6e0f4503395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:10.911413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f1db6-67e3-414d-b2ee-c6e0f4503395 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:10.911690 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f1db6-67e3-414d-b2ee-c6e0f4503395 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:10.911987 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f1db6-67e3-414d-b2ee-c6e0f4503395 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:10.912604 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f1db6-67e3-414d-b2ee-c6e0f4503395 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:10.912904 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f1db6-67e3-414d-b2ee-c6e0f4503395 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:10.914316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f1db6-67e3-414d-b2ee-c6e0f4503395 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:10.918650 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 412/845] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:11:10 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:10.919782 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877b619-d01f-4b3b-bda7-782b5519dcc3 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:10.920063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877b619-d01f-4b3b-bda7-782b5519dcc3 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:10.920334 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877b619-d01f-4b3b-bda7-782b5519dcc3 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:10.923866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 413/846] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:10.927241 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ce335-ad2c-45a8-96b2-110ac206b571 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:10.927779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ce335-ad2c-45a8-96b2-110ac206b571 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:10.928229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ce335-ad2c-45a8-96b2-110ac206b571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:10.948470 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-ad4ce335-ad2c-45a8-96b2-110ac206b571 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:10.950234 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ad4ce335-ad2c-45a8-96b2-110ac206b571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:10.977432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 434/847] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:11:10 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 11:11:10.978628 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ead224-6c94-400a-8187-f71c8bfe8b68 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:11.008639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:11.031688 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 414/848] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:11.104616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ead224-6c94-400a-8187-f71c8bfe8b68 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:11.146340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ab7a5-eb5b-4025-a036-c629c107a1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:11.146976 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ab7a5-eb5b-4025-a036-c629c107a1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:11.147253 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ab7a5-eb5b-4025-a036-c629c107a1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:11.147521 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ab7a5-eb5b-4025-a036-c629c107a1c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:11.150087 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aadc5a-08c6-4322-aaf6-55d3ac829a34 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:11.150378 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aadc5a-08c6-4322-aaf6-55d3ac829a34 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:11.150639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aadc5a-08c6-4322-aaf6-55d3ac829a34 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:11.152151 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 415/849] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:11:11.157410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b211a2-41d8-4901-aad0-7f809fa5dc55 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:11.157699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b211a2-41d8-4901-aad0-7f809fa5dc55 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:11.157974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b211a2-41d8-4901-aad0-7f809fa5dc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:11.158720 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ab7a5-eb5b-4025-a036-c629c107a1c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:11.160277 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ab7a5-eb5b-4025-a036-c629c107a1c4 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:11.176784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-b95ab7a5-eb5b-4025-a036-c629c107a1c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:11.176784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:11.176784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:11.176784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:11.176784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:11.176784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:11.176784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:11.176784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:11.176784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:11.176784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:11.176784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:11.176784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:11.176784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:11.176784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:11.176784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:11.176784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:11.177782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:11.177782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:11.177782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:11.177782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:11.177782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:11.177782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:11.177782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:11.177782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:11.177782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:11.177782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:11.177782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:11.177782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:11.177782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:11.177782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:11.177782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:11.177782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:11.177782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:11.178673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:11.178673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:11.178673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:11.178673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 435/850] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:11.181829 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-04b211a2-41d8-4901-aad0-7f809fa5dc55 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:11.182806 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ab7a5-eb5b-4025-a036-c629c107a1c4 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:11.183462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-04b211a2-41d8-4901-aad0-7f809fa5dc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:11.188885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:11.272096 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:11.280696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 416/851] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:11:11.306703 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ab7a5-eb5b-4025-a036-c629c107a1c4 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:11.320378 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b211a2-41d8-4901-aad0-7f809fa5dc55 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:11.348348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6590ed4-b0c7-456a-b365-af780e4ec448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:11.348955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6590ed4-b0c7-456a-b365-af780e4ec448 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:11.349260 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6590ed4-b0c7-456a-b365-af780e4ec448 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:11.349530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6590ed4-b0c7-456a-b365-af780e4ec448 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:11.350169 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6590ed4-b0c7-456a-b365-af780e4ec448 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:11.350469 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6590ed4-b0c7-456a-b365-af780e4ec448 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:11.351914 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6590ed4-b0c7-456a-b365-af780e4ec448 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:11.370479 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 436/852] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:11.371513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6590ed4-b0c7-456a-b365-af780e4ec448 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:11.446444 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b211a2-41d8-4901-aad0-7f809fa5dc55 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:11.482021 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14878054-fe48-413a-9719-88c69420b933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:11.482708 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14878054-fe48-413a-9719-88c69420b933 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:11.483138 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14878054-fe48-413a-9719-88c69420b933 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:11.483416 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14878054-fe48-413a-9719-88c69420b933 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:11.484136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14878054-fe48-413a-9719-88c69420b933 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:11.485678 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14878054-fe48-413a-9719-88c69420b933 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:11.494184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6590ed4-b0c7-456a-b365-af780e4ec448 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:11.495576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-14878054-fe48-413a-9719-88c69420b933 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:11:11.495576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:11.495576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:11.495576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:11.495576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:11.495576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:11.495576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:11.495576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:11.495576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:11:11.495576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:11.495576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:11.495576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:11.495576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:11.495576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:11.495576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:11.495576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:11.496612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:11.496612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:11.496612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:11.496612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:11.496612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:11.496612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:11.496612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:11.496612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:11.496612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:11.496612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:11.496612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:11.496612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:11.496612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:11.496612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:11.496612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:11.496612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:11.496612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:11.497541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:11.497541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:11.497541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:11.497541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:11.497541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:11.497541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:11.497541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:11.497541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:11.497541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:11.497541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:11.497541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:11.497541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:11.497541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:11:11.497541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:11.502036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 417/853] 10.222.0.81 () {56 vars in 1132 bytes} [Mon Sep 19 11:11:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:11.502995 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14878054-fe48-413a-9719-88c69420b933 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:11.520113 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:11.538675 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef74e6-a58c-48ea-94f9-61e848d3ab98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:11.539737 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef74e6-a58c-48ea-94f9-61e848d3ab98 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:11.540277 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef74e6-a58c-48ea-94f9-61e848d3ab98 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:11.540779 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef74e6-a58c-48ea-94f9-61e848d3ab98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:11.543898 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:11.552336 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef74e6-a58c-48ea-94f9-61e848d3ab98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:11.554030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef74e6-a58c-48ea-94f9-61e848d3ab98 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:11.569506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-fbef74e6-a58c-48ea-94f9-61e848d3ab98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:11.569506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:11.569506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:11.569506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:11.569506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:11.569506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:11.569506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:11.569506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:11.569506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:11.569506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:11.569506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:11.569506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:11.569506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:11:11.569506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:11:11.569506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:11:11.570516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:11:11.570516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:11.570516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:11.570516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:11.570516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:11.570516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:11.570516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:11.570516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:11.570516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:11.570516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:11.570516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:11.570516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:11.570516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:11.570516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:11.570516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:11.570516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:11.570516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:11.571457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:11:11.571457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:11:11.571457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:11:11.571457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:11:11.571457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:11:11.571457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:11:11.571457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:11.571457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:11.572546 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 437/854] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:11:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:11.573222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef74e6-a58c-48ea-94f9-61e848d3ab98 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:11.625671 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14878054-fe48-413a-9719-88c69420b933 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:11.665067 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbaf94-022f-4f78-b540-213236304551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:11.665714 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbaf94-022f-4f78-b540-213236304551 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:11.666034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbaf94-022f-4f78-b540-213236304551 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:11.666228 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbaf94-022f-4f78-b540-213236304551 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:11.667059 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfbaf94-022f-4f78-b540-213236304551 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:11.667168 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfbaf94-022f-4f78-b540-213236304551 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:11.670236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfbaf94-022f-4f78-b540-213236304551 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:11.677899 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 418/855] 10.222.0.81 () {56 vars in 1126 bytes} [Mon Sep 19 11:11:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:11.677899 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbaf94-022f-4f78-b540-213236304551 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:11.695744 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef74e6-a58c-48ea-94f9-61e848d3ab98 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:11.734604 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80358604-487d-4897-bc81-c4b660b24bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:11.735313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80358604-487d-4897-bc81-c4b660b24bbb None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:11.735590 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80358604-487d-4897-bc81-c4b660b24bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:11.735892 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80358604-487d-4897-bc81-c4b660b24bbb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:11.748079 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:11.748329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80358604-487d-4897-bc81-c4b660b24bbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:11:11.755309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-80358604-487d-4897-bc81-c4b660b24bbb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:11:11.755309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:11.755309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:11.755309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:11.755309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:11.755309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:11.755309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:11.755309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:11.755309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:11.755309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:11.755309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:11.755309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:11.755309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:11:11.755309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:11:11.755309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:11:11.755309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:11:11.761313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:11:11.761313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:11:11.761313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:11:11.761313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:11:11.761313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:11.761313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:11.761313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:11.761313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:11.761313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:11.761313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:11.761313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:11.761313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:11.761313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:11.761313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:11.761313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:11.761313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:11.762235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:11.762235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:11.762235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:11.762235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:11.762235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:11:11.762235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:11:11.762235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:11:11.762235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:11:11.762235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:11:11.762235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:11:11.762235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:11:11.762235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:11.762235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 438/856] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:11:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:11.762235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80358604-487d-4897-bc81-c4b660b24bbb None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:11.788849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbaf94-022f-4f78-b540-213236304551 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:11.820499 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:11.823725 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cef614-60a0-4461-85fd-61e35799b445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:11.824432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cef614-60a0-4461-85fd-61e35799b445 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:11.824707 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cef614-60a0-4461-85fd-61e35799b445 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:11.824986 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cef614-60a0-4461-85fd-61e35799b445 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:11.825673 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cef614-60a0-4461-85fd-61e35799b445 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:11.825979 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cef614-60a0-4461-85fd-61e35799b445 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:11.827409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cef614-60a0-4461-85fd-61e35799b445 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:11.839813 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 419/857] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:11:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:11.840916 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cef614-60a0-4461-85fd-61e35799b445 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:11.872939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80358604-487d-4897-bc81-c4b660b24bbb None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:11.916622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d49eaa-fa1c-4fc2-b3f3-bb2d5cc31692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:11.917357 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d49eaa-fa1c-4fc2-b3f3-bb2d5cc31692 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:11.917635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d49eaa-fa1c-4fc2-b3f3-bb2d5cc31692 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:11.918057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d49eaa-fa1c-4fc2-b3f3-bb2d5cc31692 None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:11.919421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d49eaa-fa1c-4fc2-b3f3-bb2d5cc31692 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:11.919808 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d49eaa-fa1c-4fc2-b3f3-bb2d5cc31692 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:11.921229 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d49eaa-fa1c-4fc2-b3f3-bb2d5cc31692 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:11.926157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 439/858] 10.222.0.81 () {56 vars in 1126 bytes} [Mon Sep 19 11:11:11 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:11.926966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d49eaa-fa1c-4fc2-b3f3-bb2d5cc31692 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:11.962163 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cef614-60a0-4461-85fd-61e35799b445 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:11.994292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:12.010111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6c25a-73d0-4777-b08b-847e0000462e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:12.011313 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6c25a-73d0-4777-b08b-847e0000462e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:12.011946 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6c25a-73d0-4777-b08b-847e0000462e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:12.012427 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6c25a-73d0-4777-b08b-847e0000462e None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:12.013400 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6c25a-73d0-4777-b08b-847e0000462e None admin] RBAC: Adding query filter params (name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:12.013912 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6c25a-73d0-4777-b08b-847e0000462e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:12.015721 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6c25a-73d0-4777-b08b-847e0000462e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:12.031159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 420/859] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:11:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:12.032846 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6c25a-73d0-4777-b08b-847e0000462e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:12.065640 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d49eaa-fa1c-4fc2-b3f3-bb2d5cc31692 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:12.089491 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:12.111153 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f92db2-bbbb-4b15-84a2-1a9dcc6da5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:12.112233 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f92db2-bbbb-4b15-84a2-1a9dcc6da5f2 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:12.112719 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f92db2-bbbb-4b15-84a2-1a9dcc6da5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:12.113195 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f92db2-bbbb-4b15-84a2-1a9dcc6da5f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:12.114165 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f92db2-bbbb-4b15-84a2-1a9dcc6da5f2 None admin] RBAC: Adding query filter params (role.id=753f8b35d5f0458d8f62d0ad6e28033f, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=1c8a9732807d4921a41a6a2d5019e2f5) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:12.114474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f92db2-bbbb-4b15-84a2-1a9dcc6da5f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:12.115944 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f92db2-bbbb-4b15-84a2-1a9dcc6da5f2 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:12.123369 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 440/860] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:11:11 2022] GET /identity/v3/role_assignments?user.id=1c8a9732807d4921a41a6a2d5019e2f5&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=753f8b35d5f0458d8f62d0ad6e28033f => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:12.160819 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6c25a-73d0-4777-b08b-847e0000462e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:12.202189 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fa325-3107-4097-be54-f6887d6194f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:12.203160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fa325-3107-4097-be54-f6887d6194f2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:12.204002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fa325-3107-4097-be54-f6887d6194f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:12.204474 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fa325-3107-4097-be54-f6887d6194f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:12.217468 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9fa325-3107-4097-be54-f6887d6194f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:12.219261 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9fa325-3107-4097-be54-f6887d6194f2 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:12.236167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-7a9fa325-3107-4097-be54-f6887d6194f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:12.236167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:12.236167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:12.236167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:12.236167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:12.236167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:12.236167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:12.236167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:12.236167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:12.236167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:12.236167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:12.236167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:12.236167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:12.236167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:12.236167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:12.236167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:12.237179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:12.237179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:12.237179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:12.237179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:12.237179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:12.237179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:12.237179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:12.237179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:12.237179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:12.237179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:12.237179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:12.237179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:12.237179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:12.237179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:12.237179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:12.237179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:12.237179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:12.238062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:12.238062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:12.238062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:12.240082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 421/861] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:12.249326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f92db2-bbbb-4b15-84a2-1a9dcc6da5f2 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:12.357809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f92db2-bbbb-4b15-84a2-1a9dcc6da5f2 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:12.392905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e8e45-a65c-4034-8602-64524b13773d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:12.393508 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e8e45-a65c-4034-8602-64524b13773d None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:12.393784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e8e45-a65c-4034-8602-64524b13773d None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:12.394057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e8e45-a65c-4034-8602-64524b13773d None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:12.394700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e8e45-a65c-4034-8602-64524b13773d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:12.395001 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e8e45-a65c-4034-8602-64524b13773d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:12.396394 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e8e45-a65c-4034-8602-64524b13773d None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:12.409154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 441/862] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:12.415212 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fa325-3107-4097-be54-f6887d6194f2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:12.460808 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:12.529613 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fa325-3107-4097-be54-f6887d6194f2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:12.570953 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13754f-70c8-4cbe-b397-066f237b725e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:12.571584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13754f-70c8-4cbe-b397-066f237b725e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:12.571894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13754f-70c8-4cbe-b397-066f237b725e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:12.572169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13754f-70c8-4cbe-b397-066f237b725e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:12.580325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13754f-70c8-4cbe-b397-066f237b725e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:12.581775 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13754f-70c8-4cbe-b397-066f237b725e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:12.593373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-df13754f-70c8-4cbe-b397-066f237b725e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:11:12.593373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:12.593373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:12.593373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:12.593373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:12.593373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:12.593373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:12.593373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:12.593373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:12.593373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:12.593373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:12.593373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:12.593373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:11:12.593373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:11:12.593373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:11:12.593373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:11:12.594351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:12.594351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:12.594351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:11:12.594351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:12.594351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:11:12.594351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:12.594351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:12.594351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:12.594351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:12.594351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:12.594351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:12.594351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:12.594351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:12.594351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:12.594351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:12.594351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:12.594351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:12.594351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:12.595303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:12.595303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:12.595303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:11:12.595303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:11:12.595303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:11:12.595303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:11:12.595303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:11:12.595303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:11:12.595303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:11:12.595303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:12.599250 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 422/863] 10.222.0.81 () {56 vars in 1117 bytes} [Mon Sep 19 11:11:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:12.600619 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e8e45-a65c-4034-8602-64524b13773d None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:12.645658 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:12.705981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e8e45-a65c-4034-8602-64524b13773d None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:12.744706 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14802d0e-41dd-4cb8-b6a1-7b278d158faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:12.745315 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14802d0e-41dd-4cb8-b6a1-7b278d158faa None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:12.745592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14802d0e-41dd-4cb8-b6a1-7b278d158faa None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:12.745863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14802d0e-41dd-4cb8-b6a1-7b278d158faa None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:12.746592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14802d0e-41dd-4cb8-b6a1-7b278d158faa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:12.746899 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14802d0e-41dd-4cb8-b6a1-7b278d158faa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:12.748348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14802d0e-41dd-4cb8-b6a1-7b278d158faa None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:12.771529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 442/864] 10.222.0.81 () {56 vars in 1149 bytes} [Mon Sep 19 11:11:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:12.772976 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13754f-70c8-4cbe-b397-066f237b725e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:12.880544 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13754f-70c8-4cbe-b397-066f237b725e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:12.882122 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:12.916898 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b5e7ac-be07-4378-8050-7383951606e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:12.917552 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5e7ac-be07-4378-8050-7383951606e6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:12.917838 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5e7ac-be07-4378-8050-7383951606e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:12.918115 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5e7ac-be07-4378-8050-7383951606e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:12.991443 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b5e7ac-be07-4378-8050-7383951606e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:12.993520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b5e7ac-be07-4378-8050-7383951606e6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:13.110898 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 423/865] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:11:12 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:11:13.112318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14802d0e-41dd-4cb8-b6a1-7b278d158faa None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:13.158095 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcffc75-f2ca-4a41-a719-c8504b14ab18 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:13.158286 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcffc75-f2ca-4a41-a719-c8504b14ab18 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:13.158558 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcffc75-f2ca-4a41-a719-c8504b14ab18 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:13.160200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 424/866] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:11:13.166850 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae377818-6c49-4a23-a215-68f93aa043d2 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:13.167128 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae377818-6c49-4a23-a215-68f93aa043d2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:13.167399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae377818-6c49-4a23-a215-68f93aa043d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:13.173956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:13.193901 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-ae377818-6c49-4a23-a215-68f93aa043d2 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:13.198049 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ae377818-6c49-4a23-a215-68f93aa043d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:13.249035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:13.254696 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14802d0e-41dd-4cb8-b6a1-7b278d158faa None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:13.280190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 425/867] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:11:13.294803 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d520f9-e32e-49f4-917b-a7a61ebd1538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:13.295747 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d520f9-e32e-49f4-917b-a7a61ebd1538 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:13.296501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d520f9-e32e-49f4-917b-a7a61ebd1538 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:13.296990 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d520f9-e32e-49f4-917b-a7a61ebd1538 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:13.308729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d520f9-e32e-49f4-917b-a7a61ebd1538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:13.310427 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d520f9-e32e-49f4-917b-a7a61ebd1538 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:13.325308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-a0d520f9-e32e-49f4-917b-a7a61ebd1538 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:13.325308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:13.325308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:13.325308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:13.325308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:13.325308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:13.325308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:13.325308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:13.325308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:13.325308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:13.325308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:13.325308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:13.325308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:13.325308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:13.325308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:13.325308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:13.326334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:13.326334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:13.326334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:13.326334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:13.326334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:13.326334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:13.326334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:13.326334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:13.326334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:13.326334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:13.326334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:13.326334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:13.326334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:13.326334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:13.326334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:13.326334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:13.326334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:13.327318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:13.327318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:13.327318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:13.333937 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 443/868] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:13.335507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae377818-6c49-4a23-a215-68f93aa043d2 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:13.437696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae377818-6c49-4a23-a215-68f93aa043d2 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:13.468052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:13.473879 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b0936-8636-4d5e-b861-2f17cdec3fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:13.474749 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b0936-8636-4d5e-b861-2f17cdec3fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:13.475242 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b0936-8636-4d5e-b861-2f17cdec3fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:13.475733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b0936-8636-4d5e-b861-2f17cdec3fc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:13.477140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b0936-8636-4d5e-b861-2f17cdec3fc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:13.477658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b0936-8636-4d5e-b861-2f17cdec3fc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:13.479240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b0936-8636-4d5e-b861-2f17cdec3fc1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:13.495940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 426/869] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:13.497477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d520f9-e32e-49f4-917b-a7a61ebd1538 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:13.545394 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b0936-8636-4d5e-b861-2f17cdec3fc1 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:13.605155 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d520f9-e32e-49f4-917b-a7a61ebd1538 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:13.641153 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f61925-d320-43d2-9018-05b3f2c51927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:13.642169 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f61925-d320-43d2-9018-05b3f2c51927 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:13.642653 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f61925-d320-43d2-9018-05b3f2c51927 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:13.643132 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f61925-d320-43d2-9018-05b3f2c51927 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:13.650717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f61925-d320-43d2-9018-05b3f2c51927 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:13.652438 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f61925-d320-43d2-9018-05b3f2c51927 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:13.664661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-b7f61925-d320-43d2-9018-05b3f2c51927 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:13.664661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:13.664661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:13.664661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:13.664661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:13.664661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:13.664661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:13.664661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:13.664661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:13.664661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:13.664661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:13.664661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:13.664661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:11:13.664661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:11:13.664661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:11:13.665582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:11:13.665582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:13.665582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:13.665582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:13.665582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:13.665582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:13.665582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:13.665582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:13.665582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:13.665582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:13.665582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:13.665582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:13.665582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:13.665582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:13.665582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:13.665582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:13.665582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:13.666475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:11:13.666475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:11:13.666475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:11:13.666475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:11:13.666475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:11:13.666475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:11:13.666475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:13.666475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:13.668913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 444/870] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:11:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:13.675011 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f61925-d320-43d2-9018-05b3f2c51927 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:13.677672 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b0936-8636-4d5e-b861-2f17cdec3fc1 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:13.692436 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:13.724787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e22d5-4839-401f-908a-883bc8e60863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:13.725955 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e22d5-4839-401f-908a-883bc8e60863 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:13.726314 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e22d5-4839-401f-908a-883bc8e60863 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:13.726712 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e22d5-4839-401f-908a-883bc8e60863 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:13.727453 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e22d5-4839-401f-908a-883bc8e60863 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:13.729633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e22d5-4839-401f-908a-883bc8e60863 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:13.756062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 427/871] 10.222.0.81 () {60 vars in 1141 bytes} [Mon Sep 19 11:11:13 2022] POST /identity/v3/services => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:11:13.800458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f61925-d320-43d2-9018-05b3f2c51927 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:13.849418 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0de92-bd6d-42ec-9bfb-5aef1ce3135a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:13.850039 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0de92-bd6d-42ec-9bfb-5aef1ce3135a None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:13.850318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0de92-bd6d-42ec-9bfb-5aef1ce3135a None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:13.850591 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0de92-bd6d-42ec-9bfb-5aef1ce3135a None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:13.851294 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0de92-bd6d-42ec-9bfb-5aef1ce3135a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:13.851635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0de92-bd6d-42ec-9bfb-5aef1ce3135a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:13.853122 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0de92-bd6d-42ec-9bfb-5aef1ce3135a None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:13.871542 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 445/872] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:11:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:13.876905 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e22d5-4839-401f-908a-883bc8e60863 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:13.939696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:13.994622 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e22d5-4839-401f-908a-883bc8e60863 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:14.030320 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb1953-5eae-4506-88ba-300d536b1bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:14.031282 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb1953-5eae-4506-88ba-300d536b1bbe None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:14.031716 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb1953-5eae-4506-88ba-300d536b1bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:14.032166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb1953-5eae-4506-88ba-300d536b1bbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:14.033040 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb1953-5eae-4506-88ba-300d536b1bbe None admin] RBAC: Adding query filter params (role.id=af9d39307d714527b14b2045ab466e7a, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=06d279b488d148428b7a3c35a9786ffe) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:14.033492 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb1953-5eae-4506-88ba-300d536b1bbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:14.035080 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb1953-5eae-4506-88ba-300d536b1bbe None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:14.042047 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 428/873] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:11:13 2022] GET /identity/v3/role_assignments?user.id=06d279b488d148428b7a3c35a9786ffe&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=af9d39307d714527b14b2045ab466e7a => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:14.250505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605a963-1134-4445-a90c-2df9322e607c None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:14.250688 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605a963-1134-4445-a90c-2df9322e607c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:14.250967 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605a963-1134-4445-a90c-2df9322e607c None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:14.252596 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 446/874] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:14.258098 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66af3c5-92f7-477b-98ad-111be77a95f2 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:14.258388 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66af3c5-92f7-477b-98ad-111be77a95f2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:14.258667 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66af3c5-92f7-477b-98ad-111be77a95f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:14.278413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c66af3c5-92f7-477b-98ad-111be77a95f2 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:14.279624 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c66af3c5-92f7-477b-98ad-111be77a95f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:14.350720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 429/875] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:14 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:14.506242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b3bd5-583d-4d78-9854-c66be7f32146 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:14.506433 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b3bd5-583d-4d78-9854-c66be7f32146 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:14.506708 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b3bd5-583d-4d78-9854-c66be7f32146 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:14.508386 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 447/876] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:14.513695 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a8d35-8aa7-40e9-a1f5-a70164eecd0c None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:14.513988 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a8d35-8aa7-40e9-a1f5-a70164eecd0c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:14.514263 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a8d35-8aa7-40e9-a1f5-a70164eecd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:14.533080 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9a8a8d35-8aa7-40e9-a1f5-a70164eecd0c None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:14.534274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9a8a8d35-8aa7-40e9-a1f5-a70164eecd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:14.579282 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:14.609335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 430/877] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:14 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:14.628679 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b3bd5-583d-4d78-9854-c66be7f32146 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:14.710561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:14.745973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b3bd5-583d-4d78-9854-c66be7f32146 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:14.783079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a15ec-2dec-47f8-89cf-80522eea965a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:14.783652 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a15ec-2dec-47f8-89cf-80522eea965a None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:14.783962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a15ec-2dec-47f8-89cf-80522eea965a None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:14.784231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a15ec-2dec-47f8-89cf-80522eea965a None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:14.784988 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a15ec-2dec-47f8-89cf-80522eea965a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:14.786400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a15ec-2dec-47f8-89cf-80522eea965a None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:14.805381 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 448/878] 10.222.0.81 () {60 vars in 1141 bytes} [Mon Sep 19 11:11:14 2022] POST /identity/v3/services => generated 269 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:11:14.893734 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a8d35-8aa7-40e9-a1f5-a70164eecd0c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:15.007465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a8d35-8aa7-40e9-a1f5-a70164eecd0c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:15.049852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d29b4c-5141-43ed-9727-a44adbb17ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:15.050409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d29b4c-5141-43ed-9727-a44adbb17ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:15.050687 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d29b4c-5141-43ed-9727-a44adbb17ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:15.050960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d29b4c-5141-43ed-9727-a44adbb17ea6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:15.051546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d29b4c-5141-43ed-9727-a44adbb17ea6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:15.052981 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d29b4c-5141-43ed-9727-a44adbb17ea6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:15.062769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-f2d29b4c-5141-43ed-9727-a44adbb17ea6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:11:15.062769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:15.062769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:15.062769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:15.062769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:15.062769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:15.062769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:15.062769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:15.062769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:11:15.062769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:15.062769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:15.062769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:15.062769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:15.062769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:15.062769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:15.062769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:15.062769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:15.063836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:15.063836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:15.063836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:15.063836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:15.063836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:15.063836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:15.063836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:15.063836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:15.063836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:15.063836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:15.063836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:15.063836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:15.063836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:15.063836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:15.063836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:15.063836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:15.064755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:15.064755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:15.064755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:15.064755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:15.064755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:15.064755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:15.064755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:15.064755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:15.064755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:15.064755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:15.064755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:15.064755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:15.064755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:11:15.064755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:15.064755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 431/879] 10.222.0.81 () {56 vars in 1126 bytes} [Mon Sep 19 11:11:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:15.071847 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a15ec-2dec-47f8-89cf-80522eea965a None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:15.180348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a15ec-2dec-47f8-89cf-80522eea965a None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:15.208526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:15.219234 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24898e9c-70f7-44e4-9af8-bfef99294d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:15.219867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24898e9c-70f7-44e4-9af8-bfef99294d81 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:15.220145 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24898e9c-70f7-44e4-9af8-bfef99294d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:15.220415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24898e9c-70f7-44e4-9af8-bfef99294d81 None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:15.221162 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24898e9c-70f7-44e4-9af8-bfef99294d81 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:15.221466 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24898e9c-70f7-44e4-9af8-bfef99294d81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:15.222819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24898e9c-70f7-44e4-9af8-bfef99294d81 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:15.229260 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 449/880] 10.222.0.81 () {56 vars in 1122 bytes} [Mon Sep 19 11:11:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:15.236848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d29b4c-5141-43ed-9727-a44adbb17ea6 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:15.274940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:15.344922 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d29b4c-5141-43ed-9727-a44adbb17ea6 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:15.382272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8a9c8-edd0-45e2-8e28-21783d05bac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:15.383366 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8a9c8-edd0-45e2-8e28-21783d05bac3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:15.383905 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8a9c8-edd0-45e2-8e28-21783d05bac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:15.384388 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8a9c8-edd0-45e2-8e28-21783d05bac3 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:15.385217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8a9c8-edd0-45e2-8e28-21783d05bac3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:15.385738 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8a9c8-edd0-45e2-8e28-21783d05bac3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:15.387309 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8a9c8-edd0-45e2-8e28-21783d05bac3 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:15.392452 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 432/881] 10.222.0.81 () {56 vars in 1122 bytes} [Mon Sep 19 11:11:15 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:16.100406 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ddfbd-3f10-4a5a-855c-3b5cbe2053a7 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:16.100933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ddfbd-3f10-4a5a-855c-3b5cbe2053a7 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:16.100933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ddfbd-3f10-4a5a-855c-3b5cbe2053a7 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:16.103635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 450/882] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:16.109145 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a78dcb-229a-425b-80a9-c4bcd9d3ce3c None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:16.109634 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a78dcb-229a-425b-80a9-c4bcd9d3ce3c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:16.109793 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a78dcb-229a-425b-80a9-c4bcd9d3ce3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:16.128114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-34a78dcb-229a-425b-80a9-c4bcd9d3ce3c None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:16.129325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-34a78dcb-229a-425b-80a9-c4bcd9d3ce3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:16.199321 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 433/883] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:16.395870 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f27429-bcce-4aef-ad52-0bacdd7e5fea None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:16.396181 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f27429-bcce-4aef-ad52-0bacdd7e5fea None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:16.396383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f27429-bcce-4aef-ad52-0bacdd7e5fea None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:16.397997 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 451/884] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:16.403881 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168ad38-bf9f-4449-8732-d5ac608c039a None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:16.404166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168ad38-bf9f-4449-8732-d5ac608c039a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:16.404436 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168ad38-bf9f-4449-8732-d5ac608c039a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:16.422805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c168ad38-bf9f-4449-8732-d5ac608c039a None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:16.424017 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c168ad38-bf9f-4449-8732-d5ac608c039a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:16.451897 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:16.471489 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f27429-bcce-4aef-ad52-0bacdd7e5fea None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:16.508172 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 434/885] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:16.590371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f27429-bcce-4aef-ad52-0bacdd7e5fea None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:16.629530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef40ff-d060-4c21-8f79-45f0a844e4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:16.630589 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef40ff-d060-4c21-8f79-45f0a844e4fc None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:16.631107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef40ff-d060-4c21-8f79-45f0a844e4fc None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:16.631620 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef40ff-d060-4c21-8f79-45f0a844e4fc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:16.633475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef40ff-d060-4c21-8f79-45f0a844e4fc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:16.636013 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef40ff-d060-4c21-8f79-45f0a844e4fc None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:16.645868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-96ef40ff-d060-4c21-8f79-45f0a844e4fc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:11:16.645868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:16.645868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:16.645868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:16.645868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:16.645868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:16.645868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:16.645868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:16.645868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:11:16.645868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:16.645868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:16.645868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:16.645868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:16.645868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:16.645868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:16.645868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:16.645868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:16.646908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:16.646908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:16.646908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:16.646908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:16.646908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:16.646908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:16.646908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:16.646908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:16.646908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:16.646908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:16.646908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:16.646908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:16.646908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:16.646908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:16.646908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:16.646908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:16.647829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:16.647829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:16.647829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:16.647829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:16.647829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:16.647829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:16.647829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:16.647829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:16.647829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:16.647829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:16.647829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:16.647829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:16.647829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:11:16.647829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:16.650548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 452/886] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:11:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:16.660152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168ad38-bf9f-4449-8732-d5ac608c039a None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:16.773400 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168ad38-bf9f-4449-8732-d5ac608c039a None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:16.785334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef40ff-d060-4c21-8f79-45f0a844e4fc None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:16.810155 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1902c-81ef-4a3b-9221-e34b4e881b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:16.810749 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1902c-81ef-4a3b-9221-e34b4e881b74 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:16.811026 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1902c-81ef-4a3b-9221-e34b4e881b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:16.811294 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1902c-81ef-4a3b-9221-e34b4e881b74 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:16.811944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1902c-81ef-4a3b-9221-e34b4e881b74 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:16.812246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1902c-81ef-4a3b-9221-e34b4e881b74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:16.813897 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1902c-81ef-4a3b-9221-e34b4e881b74 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:16.823218 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 435/887] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:11:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:16.825131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1902c-81ef-4a3b-9221-e34b4e881b74 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:16.855248 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:16.858470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:16.921839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef40ff-d060-4c21-8f79-45f0a844e4fc None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:16.940198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1902c-81ef-4a3b-9221-e34b4e881b74 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:16.972501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba438db-6431-471b-9967-48adac0ecc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:16.974823 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba438db-6431-471b-9967-48adac0ecc50 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:16.975439 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba438db-6431-471b-9967-48adac0ecc50 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:16.975953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba438db-6431-471b-9967-48adac0ecc50 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:16.979128 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ab74d-f525-411f-9ec2-d824db67c2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:16.979819 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ab74d-f525-411f-9ec2-d824db67c2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:16.980060 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ab74d-f525-411f-9ec2-d824db67c2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:16.980335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ab74d-f525-411f-9ec2-d824db67c2b0 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:16.980970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ab74d-f525-411f-9ec2-d824db67c2b0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:16.981274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ab74d-f525-411f-9ec2-d824db67c2b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:16.982689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ab74d-f525-411f-9ec2-d824db67c2b0 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:16.989229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 436/888] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:11:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:16.991478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba438db-6431-471b-9967-48adac0ecc50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:11:16.994903 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ab74d-f525-411f-9ec2-d824db67c2b0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:16.997082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-5ba438db-6431-471b-9967-48adac0ecc50 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:11:16.997082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:16.997082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:16.997082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:16.997082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:16.997082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:16.997082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:16.997082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:16.997082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:16.997082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:16.997082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:16.997082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:16.997082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:11:16.997082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:11:16.997082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:11:16.997082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:11:16.998035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:11:16.998035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:11:16.998035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:11:16.998035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:11:16.998035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:16.998035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:16.998035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:16.998035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:16.998035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:16.998035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:16.998035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:16.998035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:16.998035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:16.998035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:16.998035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:16.998035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:16.998888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:16.998888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:16.998888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:16.998888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:16.998888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:11:16.998888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:11:16.998888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:11:16.998888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:11:16.998888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:11:16.998888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:11:16.998888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:11:16.998888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:17.008002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 453/889] 10.222.0.81 () {56 vars in 1120 bytes} [Mon Sep 19 11:11:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:17.012152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba438db-6431-471b-9967-48adac0ecc50 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:17.109120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:17.119292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba438db-6431-471b-9967-48adac0ecc50 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:17.125016 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ab74d-f525-411f-9ec2-d824db67c2b0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:17.136704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:17.155910 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f5af9-8ea2-42fb-8e9e-239c98f45e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:17.156508 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f5af9-8ea2-42fb-8e9e-239c98f45e38 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:17.156789 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f5af9-8ea2-42fb-8e9e-239c98f45e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:17.157043 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f5af9-8ea2-42fb-8e9e-239c98f45e38 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:17.157728 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f5af9-8ea2-42fb-8e9e-239c98f45e38 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:17.158189 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f5af9-8ea2-42fb-8e9e-239c98f45e38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:17.159365 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f5af9-8ea2-42fb-8e9e-239c98f45e38 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:17.162846 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40c486-477f-4706-9c19-414982b5b805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:17.163451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40c486-477f-4706-9c19-414982b5b805 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:17.163752 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40c486-477f-4706-9c19-414982b5b805 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:17.164046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40c486-477f-4706-9c19-414982b5b805 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:17.164748 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e40c486-477f-4706-9c19-414982b5b805 None admin] RBAC: Adding query filter params (interface=public, service_id=7c7ff6c089644f33aacd2346fea5d37c, region_id=RegionOne) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:17.165050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e40c486-477f-4706-9c19-414982b5b805 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:17.166412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e40c486-477f-4706-9c19-414982b5b805 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:17.172335 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 454/890] 10.222.0.81 () {56 vars in 1115 bytes} [Mon Sep 19 11:11:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:17.172335 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58713aa2-bf11-4727-92ad-404288ab22ae None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:17.172335 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58713aa2-bf11-4727-92ad-404288ab22ae None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:17.172335 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58713aa2-bf11-4727-92ad-404288ab22ae None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:17.172335 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 455/891] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:17.175599 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58713aa2-bf11-4727-92ad-404288ab22ae None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:17.177104 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 437/892] 10.222.0.81 () {56 vars in 1251 bytes} [Mon Sep 19 11:11:16 2022] GET /identity/v3/endpoints?service_id=7c7ff6c089644f33aacd2346fea5d37c&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:17.181786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978c269-5359-4a10-9614-6a8cd4ed7013 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:17.181786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978c269-5359-4a10-9614-6a8cd4ed7013 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:17.181786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978c269-5359-4a10-9614-6a8cd4ed7013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:17.196898 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e978c269-5359-4a10-9614-6a8cd4ed7013 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:17.198093 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e978c269-5359-4a10-9614-6a8cd4ed7013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:17.265199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 438/893] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:17.276005 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978c269-5359-4a10-9614-6a8cd4ed7013 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:17.302984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58713aa2-bf11-4727-92ad-404288ab22ae None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:17.345380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5843c-d707-4d1d-88dc-12482e15c2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:17.346049 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5843c-d707-4d1d-88dc-12482e15c2ca None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:17.346326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5843c-d707-4d1d-88dc-12482e15c2ca None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:17.346603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5843c-d707-4d1d-88dc-12482e15c2ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:17.358933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca5843c-d707-4d1d-88dc-12482e15c2ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:17.360848 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca5843c-d707-4d1d-88dc-12482e15c2ca None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:17.365169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:17.380017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-eca5843c-d707-4d1d-88dc-12482e15c2ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:17.380017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:17.380017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:17.380017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:17.380017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:17.380017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:17.380017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:17.380017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:17.380017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:17.380017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:17.380017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:17.380017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:17.380017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:17.380017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:17.380017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:17.380017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:17.380984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:17.380984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:17.380984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:17.380984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:17.380984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:17.380984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:17.380984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:17.380984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:17.380984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:17.380984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:17.380984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:17.380984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:17.380984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:17.380984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:17.380984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:17.380984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:17.380984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:17.381867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:17.381867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:17.381867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:17.383920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 456/894] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:17.387752 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5843c-d707-4d1d-88dc-12482e15c2ca None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:17.391215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978c269-5359-4a10-9614-6a8cd4ed7013 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:17.423430 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:17.430406 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fd2b8-81d8-4c34-9488-89168d2467c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:17.431119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fd2b8-81d8-4c34-9488-89168d2467c6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:17.431537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fd2b8-81d8-4c34-9488-89168d2467c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:17.435797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fd2b8-81d8-4c34-9488-89168d2467c6 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:17.435797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5fd2b8-81d8-4c34-9488-89168d2467c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:17.435797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5fd2b8-81d8-4c34-9488-89168d2467c6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:17.441238 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 439/895] 10.222.0.81 () {56 vars in 1087 bytes} [Mon Sep 19 11:11:17 2022] GET /identity/v3/services => generated 1268 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:11:17.516730 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5843c-d707-4d1d-88dc-12482e15c2ca None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:17.559336 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9890c-f909-4415-bad9-bbb7c7464766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:17.560288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9890c-f909-4415-bad9-bbb7c7464766 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:17.560736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9890c-f909-4415-bad9-bbb7c7464766 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:17.561170 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9890c-f909-4415-bad9-bbb7c7464766 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:17.562021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9890c-f909-4415-bad9-bbb7c7464766 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:17.562469 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9890c-f909-4415-bad9-bbb7c7464766 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:17.564024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9890c-f909-4415-bad9-bbb7c7464766 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:17.574985 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fd2b8-81d8-4c34-9488-89168d2467c6 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:17.579000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 457/896] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:17.587909 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9890c-f909-4415-bad9-bbb7c7464766 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:17.696790 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fd2b8-81d8-4c34-9488-89168d2467c6 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:17.707044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:17.718166 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9890c-f909-4415-bad9-bbb7c7464766 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:17.733344 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042eefd2-9b81-4c1e-9743-f0f129a14760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:17.734230 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042eefd2-9b81-4c1e-9743-f0f129a14760 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:17.734514 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042eefd2-9b81-4c1e-9743-f0f129a14760 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:17.734837 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042eefd2-9b81-4c1e-9743-f0f129a14760 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:17.735419 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042eefd2-9b81-4c1e-9743-f0f129a14760 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:17.736819 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042eefd2-9b81-4c1e-9743-f0f129a14760 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:17.746753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-042eefd2-9b81-4c1e-9743-f0f129a14760 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:11:17.746753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:17.746753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:17.746753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:17.746753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:17.746753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:17.746753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:17.746753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:17.746753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:11:17.746753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:17.746753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:17.746753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:17.746753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:17.746753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:17.746753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:17.746753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:17.747718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:17.747718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:17.747718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:17.747718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:17.747718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:17.747718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:17.747718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:17.747718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:17.747718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:17.747718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:17.747718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:17.747718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:17.747718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:17.747718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:17.747718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:17.747718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:17.747718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:17.748655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:17.748655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:17.748655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:17.748655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:17.748655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:17.748655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:17.748655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:17.748655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:17.748655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:17.748655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:17.748655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:17.748655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:17.748655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:11:17.748655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:17.752784 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 440/897] 10.222.0.81 () {56 vars in 1132 bytes} [Mon Sep 19 11:11:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:17.753797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056a803-92d0-49cc-91d2-00a2f82f9eff None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:17.754127 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056a803-92d0-49cc-91d2-00a2f82f9eff None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:17.754353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056a803-92d0-49cc-91d2-00a2f82f9eff None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:17.755891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 441/898] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:17.756452 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056a803-92d0-49cc-91d2-00a2f82f9eff None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:17.757195 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20a0cb-d557-4495-bad9-f15781b44953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:17.758312 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20a0cb-d557-4495-bad9-f15781b44953 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:17.758590 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20a0cb-d557-4495-bad9-f15781b44953 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:17.758860 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20a0cb-d557-4495-bad9-f15781b44953 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:17.767332 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20a0cb-d557-4495-bad9-f15781b44953 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:17.768793 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20a0cb-d557-4495-bad9-f15781b44953 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:17.769940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:17.781946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-9e20a0cb-d557-4495-bad9-f15781b44953 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:11:17.781946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:17.781946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:17.781946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:17.781946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:17.781946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:17.781946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:17.781946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:17.781946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:17.781946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:17.781946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:17.781946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:17.781946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:11:17.781946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:11:17.781946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:11:17.781946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:11:17.783125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:17.783125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:17.783125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:11:17.783125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:17.783125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:11:17.783125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:17.783125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:17.783125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:17.783125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:17.783125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:17.783125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:17.783125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:17.783125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:17.783125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:17.783125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:17.783125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:17.783125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:17.783125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:17.784352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:17.784352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:17.784352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:11:17.784352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:11:17.784352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:11:17.784352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:11:17.784352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:11:17.784352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:11:17.784352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:11:17.784352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:17.784352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 458/899] 10.222.0.81 () {56 vars in 1117 bytes} [Mon Sep 19 11:11:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:17.784352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abef539-f8cf-4f4d-ab26-7c496df65bed None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:17.785222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abef539-f8cf-4f4d-ab26-7c496df65bed None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:17.785222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abef539-f8cf-4f4d-ab26-7c496df65bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:17.805722 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-7abef539-f8cf-4f4d-ab26-7c496df65bed None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:17.806945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-7abef539-f8cf-4f4d-ab26-7c496df65bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:17.870890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056a803-92d0-49cc-91d2-00a2f82f9eff None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:17.880258 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 459/900] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:17.880947 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abef539-f8cf-4f4d-ab26-7c496df65bed None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:17.923319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f927467-8972-476c-b61e-29d0486216b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:17.924401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f927467-8972-476c-b61e-29d0486216b8 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:17.924904 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f927467-8972-476c-b61e-29d0486216b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:17.925442 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f927467-8972-476c-b61e-29d0486216b8 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:17.926272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f927467-8972-476c-b61e-29d0486216b8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:17.927279 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f927467-8972-476c-b61e-29d0486216b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:17.929559 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f927467-8972-476c-b61e-29d0486216b8 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:17.941093 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 442/901] 10.222.0.81 () {56 vars in 1126 bytes} [Mon Sep 19 11:11:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:17.942610 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f927467-8972-476c-b61e-29d0486216b8 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:17.996264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:18.020409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abef539-f8cf-4f4d-ab26-7c496df65bed None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:18.036009 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:18.059449 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2955d06-816f-41a3-97f4-1353ce02552a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:18.060488 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2955d06-816f-41a3-97f4-1353ce02552a None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:18.060970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2955d06-816f-41a3-97f4-1353ce02552a None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:18.061455 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2955d06-816f-41a3-97f4-1353ce02552a None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:18.062388 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2955d06-816f-41a3-97f4-1353ce02552a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:18.062894 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2955d06-816f-41a3-97f4-1353ce02552a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:18.064548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2955d06-816f-41a3-97f4-1353ce02552a None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:18.069510 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f927467-8972-476c-b61e-29d0486216b8 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:18.087752 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 460/902] 10.222.0.81 () {56 vars in 1149 bytes} [Mon Sep 19 11:11:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:18.095552 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2955d06-816f-41a3-97f4-1353ce02552a None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:18.112193 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6672e-2894-4dc7-8b22-dda323642169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:18.113273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6672e-2894-4dc7-8b22-dda323642169 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:18.113752 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6672e-2894-4dc7-8b22-dda323642169 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:18.114280 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6672e-2894-4dc7-8b22-dda323642169 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:18.115092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6672e-2894-4dc7-8b22-dda323642169 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:18.115669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6672e-2894-4dc7-8b22-dda323642169 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:18.117923 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6672e-2894-4dc7-8b22-dda323642169 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:18.128436 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 443/903] 10.222.0.81 () {56 vars in 1126 bytes} [Mon Sep 19 11:11:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:18.130244 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6672e-2894-4dc7-8b22-dda323642169 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:18.205057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2955d06-816f-41a3-97f4-1353ce02552a None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:18.239056 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6672e-2894-4dc7-8b22-dda323642169 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:18.240652 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58e3eb-d138-4ca9-866b-c100ae891be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:18.242506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58e3eb-d138-4ca9-866b-c100ae891be6 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:18.242506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58e3eb-d138-4ca9-866b-c100ae891be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:18.242506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58e3eb-d138-4ca9-866b-c100ae891be6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:18.251226 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:18.278910 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638672e8-4c1f-4e72-83cd-d9b642c5f849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:18.279520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638672e8-4c1f-4e72-83cd-d9b642c5f849 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:18.279835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638672e8-4c1f-4e72-83cd-d9b642c5f849 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:18.280113 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638672e8-4c1f-4e72-83cd-d9b642c5f849 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:18.280814 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638672e8-4c1f-4e72-83cd-d9b642c5f849 None admin] RBAC: Adding query filter params (interface=public, service_id=391b4208025e4d1988d313c26205ff1f, region_id=RegionOne) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:18.281145 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638672e8-4c1f-4e72-83cd-d9b642c5f849 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:18.282499 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638672e8-4c1f-4e72-83cd-d9b642c5f849 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:18.287645 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 444/904] 10.222.0.81 () {56 vars in 1251 bytes} [Mon Sep 19 11:11:18 2022] GET /identity/v3/endpoints?service_id=391b4208025e4d1988d313c26205ff1f&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:18.288454 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638672e8-4c1f-4e72-83cd-d9b642c5f849 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:18.291270 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:18.324973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a58e3eb-d138-4ca9-866b-c100ae891be6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:18.326424 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a58e3eb-d138-4ca9-866b-c100ae891be6 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:18.410547 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638672e8-4c1f-4e72-83cd-d9b642c5f849 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:18.430766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 461/905] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:11:18 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:11:18.431412 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58e3eb-d138-4ca9-866b-c100ae891be6 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:18.451118 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5cca2-0a77-4e4c-b316-8c22c06ebf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:18.452163 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5cca2-0a77-4e4c-b316-8c22c06ebf6c None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:18.452541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5cca2-0a77-4e4c-b316-8c22c06ebf6c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:18.452823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5cca2-0a77-4e4c-b316-8c22c06ebf6c None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:18.453887 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5cca2-0a77-4e4c-b316-8c22c06ebf6c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:18.455267 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5cca2-0a77-4e4c-b316-8c22c06ebf6c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:18.470074 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 445/906] 10.222.0.81 () {60 vars in 1141 bytes} [Mon Sep 19 11:11:18 2022] POST /identity/v3/services => generated 241 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:11:18.470868 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5cca2-0a77-4e4c-b316-8c22c06ebf6c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:18.473896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:18.560298 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:18.568379 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58e3eb-d138-4ca9-866b-c100ae891be6 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:18.608398 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5cca2-0a77-4e4c-b316-8c22c06ebf6c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:18.616094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45a5bd-cd02-4523-80e2-0d61511887db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:18.617030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45a5bd-cd02-4523-80e2-0d61511887db None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:18.617461 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45a5bd-cd02-4523-80e2-0d61511887db None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:18.617880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45a5bd-cd02-4523-80e2-0d61511887db None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:18.618635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c45a5bd-cd02-4523-80e2-0d61511887db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:18.620876 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c45a5bd-cd02-4523-80e2-0d61511887db None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:18.635965 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 462/907] 10.222.0.81 () {56 vars in 1087 bytes} [Mon Sep 19 11:11:18 2022] GET /identity/v3/services => generated 1497 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:11:18.657582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d066e07-4409-4c70-948f-53e49627677e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:18.658238 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d066e07-4409-4c70-948f-53e49627677e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:18.658520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d066e07-4409-4c70-948f-53e49627677e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:18.658794 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d066e07-4409-4c70-948f-53e49627677e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:18.680901 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d066e07-4409-4c70-948f-53e49627677e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:18.682458 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d066e07-4409-4c70-948f-53e49627677e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:18.700372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-3d066e07-4409-4c70-948f-53e49627677e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:18.700372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:18.700372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:18.700372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:18.700372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:18.700372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:18.700372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:18.700372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:18.700372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:18.700372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:18.700372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:18.700372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:18.700372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:18.700372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:18.700372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:18.700372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:18.701411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:18.701411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:18.701411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:18.701411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:18.701411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:18.701411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:18.701411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:18.701411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:18.701411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:18.701411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:18.701411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:18.701411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:18.701411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:18.701411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:18.701411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:18.701411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:18.701411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:18.703015 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:18.703015 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:18.703015 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:18.703015 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 446/908] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:18.708012 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45a5bd-cd02-4523-80e2-0d61511887db None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:18.825764 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45a5bd-cd02-4523-80e2-0d61511887db None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:18.827417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:18.863845 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd1222-1c0a-4bcd-b8e2-3b854e743c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:18.864447 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd1222-1c0a-4bcd-b8e2-3b854e743c5b None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:18.864720 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd1222-1c0a-4bcd-b8e2-3b854e743c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:18.864989 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd1222-1c0a-4bcd-b8e2-3b854e743c5b None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:18.865624 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd1222-1c0a-4bcd-b8e2-3b854e743c5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:18.865921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd1222-1c0a-4bcd-b8e2-3b854e743c5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:18.867297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd1222-1c0a-4bcd-b8e2-3b854e743c5b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:18.879378 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 463/909] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:18.885086 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d066e07-4409-4c70-948f-53e49627677e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:18.994486 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d066e07-4409-4c70-948f-53e49627677e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:19.005491 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:19.033260 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcfea7-b415-4512-a10f-0af06836a614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:19.033896 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcfea7-b415-4512-a10f-0af06836a614 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:19.034179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcfea7-b415-4512-a10f-0af06836a614 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:19.034454 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcfea7-b415-4512-a10f-0af06836a614 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:19.041657 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dcfea7-b415-4512-a10f-0af06836a614 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:19.043106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dcfea7-b415-4512-a10f-0af06836a614 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:19.055171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-d9dcfea7-b415-4512-a10f-0af06836a614 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:19.055171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:19.055171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:19.055171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:19.055171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:19.055171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:19.055171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:19.055171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:19.055171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:19.055171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:19.055171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:19.055171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:19.055171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:11:19.055171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:11:19.055171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:11:19.056152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:11:19.056152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:19.056152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:19.056152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:19.056152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:19.056152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:19.056152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:19.056152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:19.056152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:19.056152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:19.056152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:19.056152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:19.056152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:19.056152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:19.056152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:19.056152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:19.056152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:19.057140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:11:19.057140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:11:19.057140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:11:19.057140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:11:19.057140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:11:19.057140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:11:19.057140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:19.057140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:19.057140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 447/910] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:11:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:19.063923 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd1222-1c0a-4bcd-b8e2-3b854e743c5b None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:19.170308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd1222-1c0a-4bcd-b8e2-3b854e743c5b None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:19.206701 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2aef5-1032-409b-a1ac-1128d9474807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:19.208136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2aef5-1032-409b-a1ac-1128d9474807 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:19.208630 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2aef5-1032-409b-a1ac-1128d9474807 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:19.209107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2aef5-1032-409b-a1ac-1128d9474807 None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:19.210018 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2aef5-1032-409b-a1ac-1128d9474807 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:19.210608 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2aef5-1032-409b-a1ac-1128d9474807 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:19.212271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2aef5-1032-409b-a1ac-1128d9474807 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:19.225494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 464/911] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:11:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:19.232204 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcfea7-b415-4512-a10f-0af06836a614 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:19.341298 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcfea7-b415-4512-a10f-0af06836a614 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:19.379486 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eddecd-5c77-4cfb-a30e-38bcd70b79e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:19.380133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eddecd-5c77-4cfb-a30e-38bcd70b79e8 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:19.380423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eddecd-5c77-4cfb-a30e-38bcd70b79e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:19.380709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eddecd-5c77-4cfb-a30e-38bcd70b79e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:19.381414 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eddecd-5c77-4cfb-a30e-38bcd70b79e8 None admin] RBAC: Adding query filter params (role.id=7689e410cf25472397444b35809be2c7, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=06d279b488d148428b7a3c35a9786ffe) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:19.381731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eddecd-5c77-4cfb-a30e-38bcd70b79e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:19.383727 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eddecd-5c77-4cfb-a30e-38bcd70b79e8 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:19.391808 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 448/912] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:11:19 2022] GET /identity/v3/role_assignments?user.id=06d279b488d148428b7a3c35a9786ffe&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=7689e410cf25472397444b35809be2c7 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:19.740913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc419d83-caa5-4a5f-95c3-18fae30158da None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:19.741581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc419d83-caa5-4a5f-95c3-18fae30158da None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:19.742095 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc419d83-caa5-4a5f-95c3-18fae30158da None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:19.743907 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 465/913] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:11:19.749727 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979b445-c6a6-4fe9-a82d-37451e6e367f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:19.750264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979b445-c6a6-4fe9-a82d-37451e6e367f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:19.750763 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979b445-c6a6-4fe9-a82d-37451e6e367f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:19.769656 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e979b445-c6a6-4fe9-a82d-37451e6e367f None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:19.771171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e979b445-c6a6-4fe9-a82d-37451e6e367f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:19.800055 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:19.843435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 449/914] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:20.108791 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc419d83-caa5-4a5f-95c3-18fae30158da None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:20.135722 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:20.217872 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc419d83-caa5-4a5f-95c3-18fae30158da None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:20.258696 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f22bb-3ff9-486c-896d-8c5f25fa9b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:20.259330 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f22bb-3ff9-486c-896d-8c5f25fa9b25 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:20.259609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f22bb-3ff9-486c-896d-8c5f25fa9b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:20.259913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f22bb-3ff9-486c-896d-8c5f25fa9b25 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:20.260499 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f22bb-3ff9-486c-896d-8c5f25fa9b25 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:20.261855 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f22bb-3ff9-486c-896d-8c5f25fa9b25 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:20.271203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-ca9f22bb-3ff9-486c-896d-8c5f25fa9b25 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:11:20.271203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:20.271203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:20.271203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:20.271203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:20.271203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:20.271203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:20.271203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:20.271203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:11:20.271203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:20.271203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:20.271203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:20.271203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:20.271203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:20.271203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:20.271203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:20.271203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:20.272531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:20.272531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:20.272531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:20.272531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:20.272531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:20.272531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:20.272531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:20.272531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:20.272531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:20.272531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:20.272531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:20.272531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:20.272531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:20.272531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:20.272531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:20.272531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:20.273464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:20.273464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:20.273464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:20.273464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:20.273464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:20.273464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:20.273464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:20.273464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:20.273464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:20.273464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:20.273464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:20.273464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:20.273464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 11:11:20.273464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:20.273464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 466/915] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:11:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:20.281181 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979b445-c6a6-4fe9-a82d-37451e6e367f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:20.388500 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979b445-c6a6-4fe9-a82d-37451e6e367f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:20.429319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f7958-f16e-483e-981c-5e347049b829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:20.429932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f7958-f16e-483e-981c-5e347049b829 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:20.430207 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f7958-f16e-483e-981c-5e347049b829 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:20.430685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f7958-f16e-483e-981c-5e347049b829 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:20.431474 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f7958-f16e-483e-981c-5e347049b829 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:20.431787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f7958-f16e-483e-981c-5e347049b829 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:20.433256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f7958-f16e-483e-981c-5e347049b829 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:20.438607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 450/916] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:11:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:20.444228 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f22bb-3ff9-486c-896d-8c5f25fa9b25 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:20.536510 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:20.553790 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f22bb-3ff9-486c-896d-8c5f25fa9b25 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:20.592061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c230619-d6cc-45d2-a517-6bec02168781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:20.592668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c230619-d6cc-45d2-a517-6bec02168781 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:20.592959 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c230619-d6cc-45d2-a517-6bec02168781 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:20.593231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c230619-d6cc-45d2-a517-6bec02168781 None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:20.593852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c230619-d6cc-45d2-a517-6bec02168781 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:20.594164 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c230619-d6cc-45d2-a517-6bec02168781 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:20.595526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c230619-d6cc-45d2-a517-6bec02168781 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:20.600883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 467/917] 10.222.0.81 () {56 vars in 1114 bytes} [Mon Sep 19 11:11:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:20.607014 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f7958-f16e-483e-981c-5e347049b829 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:20.639609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:20.729574 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f7958-f16e-483e-981c-5e347049b829 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:20.766206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845179b-d603-4823-b1ed-08067f91108d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:20.767208 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845179b-d603-4823-b1ed-08067f91108d None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:20.767709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845179b-d603-4823-b1ed-08067f91108d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:20.768232 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845179b-d603-4823-b1ed-08067f91108d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:20.769097 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845179b-d603-4823-b1ed-08067f91108d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:20.770666 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845179b-d603-4823-b1ed-08067f91108d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:20.793326 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 451/918] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:11:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:11:20.803452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80dd88e-d7a3-41a8-852e-e3374755823d None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:20.803809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80dd88e-d7a3-41a8-852e-e3374755823d None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:20.804070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80dd88e-d7a3-41a8-852e-e3374755823d None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:20.805738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 468/919] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:20.811482 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb2785-fd2c-4ace-be3c-09093fe92c33 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:20.811851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb2785-fd2c-4ace-be3c-09093fe92c33 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:20.812062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb2785-fd2c-4ace-be3c-09093fe92c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:20.833157 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-57bb2785-fd2c-4ace-be3c-09093fe92c33 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:20.834412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-57bb2785-fd2c-4ace-be3c-09093fe92c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:20.925850 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 452/920] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:20.973694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4988c-6123-4e9b-8565-d06ee44b167c None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:20.974014 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4988c-6123-4e9b-8565-d06ee44b167c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:20.974245 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4988c-6123-4e9b-8565-d06ee44b167c None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:20.975914 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 469/921] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:20.981893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a203793-fbf7-4843-bedc-951214ae4aad None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:20.982183 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a203793-fbf7-4843-bedc-951214ae4aad None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:20.982453 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a203793-fbf7-4843-bedc-951214ae4aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:21.001090 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6a203793-fbf7-4843-bedc-951214ae4aad None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:21.002275 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6a203793-fbf7-4843-bedc-951214ae4aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:21.005203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:21.079897 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 453/922] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:21.195594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4988c-6123-4e9b-8565-d06ee44b167c None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:21.297060 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4988c-6123-4e9b-8565-d06ee44b167c None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:21.332536 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34e9b1-f63e-4248-b776-1d5f472a130a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:21.333531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34e9b1-f63e-4248-b776-1d5f472a130a None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:21.334024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34e9b1-f63e-4248-b776-1d5f472a130a None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:21.334503 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34e9b1-f63e-4248-b776-1d5f472a130a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:21.335341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34e9b1-f63e-4248-b776-1d5f472a130a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:21.336942 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34e9b1-f63e-4248-b776-1d5f472a130a None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:21.345883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-5d34e9b1-f63e-4248-b776-1d5f472a130a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:11:21.345883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:21.345883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:21.345883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:21.345883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:21.345883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:21.345883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:21.345883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:21.345883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:11:21.345883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:21.345883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:21.345883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:21.345883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:21.345883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:21.345883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:21.345883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:21.345883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:21.347002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:21.347002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:21.347002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:21.347002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:21.347002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:21.347002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:21.347002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:21.347002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:21.347002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:21.347002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:21.347002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:21.347002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:21.347002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:21.347002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:21.347002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:21.347002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:21.348036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:21.348036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:21.348036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:21.348036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:21.348036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:21.348036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:21.348036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:21.348036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:21.348036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:21.348036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:21.348036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:21.348036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:21.348036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:11:21.348036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:21.348779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a203793-fbf7-4843-bedc-951214ae4aad None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:21.349674 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 470/923] 10.222.0.81 () {56 vars in 1126 bytes} [Mon Sep 19 11:11:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:21.355614 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34e9b1-f63e-4248-b776-1d5f472a130a None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:21.376501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:21.456036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a203793-fbf7-4843-bedc-951214ae4aad None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:21.469031 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34e9b1-f63e-4248-b776-1d5f472a130a None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:21.490667 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5008d8-302e-4309-8611-255274f7eefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:21.491294 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5008d8-302e-4309-8611-255274f7eefc None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:21.491566 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5008d8-302e-4309-8611-255274f7eefc None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:21.491884 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5008d8-302e-4309-8611-255274f7eefc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:21.492458 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5008d8-302e-4309-8611-255274f7eefc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:21.494083 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5008d8-302e-4309-8611-255274f7eefc None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:21.502743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-9e5008d8-302e-4309-8611-255274f7eefc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:11:21.502743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:21.502743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:21.502743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:21.502743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:21.502743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:21.502743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:21.502743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:21.502743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:11:21.502743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:21.502743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:21.502743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:21.502743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:21.502743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:21.502743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:21.502743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:21.503694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:21.503694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:21.503694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:21.503694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:21.503694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:21.503694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:21.503694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:21.503694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:21.503694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:21.503694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:21.503694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:21.503694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:21.503694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:21.503694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:21.503694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:21.503694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:21.503694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:21.506045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:21.506045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:21.506045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:21.506045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:21.506045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:21.506045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:21.506045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:21.506045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:21.506045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:21.506045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:21.506045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:21.506045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:21.506045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 11:11:21.506045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:21.506045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 454/924] 10.222.0.81 () {56 vars in 1132 bytes} [Mon Sep 19 11:11:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:21.512782 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5008d8-302e-4309-8611-255274f7eefc None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:21.517339 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:21.519369 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd08924-8310-411b-a27f-92bb31f7d561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:21.520393 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd08924-8310-411b-a27f-92bb31f7d561 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:21.520872 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd08924-8310-411b-a27f-92bb31f7d561 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:21.521338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd08924-8310-411b-a27f-92bb31f7d561 None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:21.522174 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd08924-8310-411b-a27f-92bb31f7d561 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:21.522670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd08924-8310-411b-a27f-92bb31f7d561 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:21.525095 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd08924-8310-411b-a27f-92bb31f7d561 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:21.535504 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 471/925] 10.222.0.81 () {56 vars in 1122 bytes} [Mon Sep 19 11:11:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:21.536730 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd08924-8310-411b-a27f-92bb31f7d561 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:21.626725 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:21.651751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd08924-8310-411b-a27f-92bb31f7d561 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:21.653452 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5008d8-302e-4309-8611-255274f7eefc None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:21.694431 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be9d29-9c2b-4019-8d15-67541a0f9545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:21.695630 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81890e6d-2b08-421d-8110-d9c2f287374f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:21.696205 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be9d29-9c2b-4019-8d15-67541a0f9545 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:21.696315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81890e6d-2b08-421d-8110-d9c2f287374f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:21.696527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81890e6d-2b08-421d-8110-d9c2f287374f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:21.696798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81890e6d-2b08-421d-8110-d9c2f287374f None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:21.697188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be9d29-9c2b-4019-8d15-67541a0f9545 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:21.697426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81890e6d-2b08-421d-8110-d9c2f287374f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:21.697729 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81890e6d-2b08-421d-8110-d9c2f287374f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:21.698046 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be9d29-9c2b-4019-8d15-67541a0f9545 None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:21.698853 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be9d29-9c2b-4019-8d15-67541a0f9545 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:21.699105 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81890e6d-2b08-421d-8110-d9c2f287374f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:21.699574 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be9d29-9c2b-4019-8d15-67541a0f9545 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:21.701337 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be9d29-9c2b-4019-8d15-67541a0f9545 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:21.704397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 455/926] 10.222.0.81 () {56 vars in 1126 bytes} [Mon Sep 19 11:11:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:21.705686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c989d-1fbc-4c73-a2af-53af8c96fa5f None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:21.705969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c989d-1fbc-4c73-a2af-53af8c96fa5f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:21.706289 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c989d-1fbc-4c73-a2af-53af8c96fa5f None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:21.707560 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 456/927] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:21.715794 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c989d-1fbc-4c73-a2af-53af8c96fa5f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:21.715933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 472/928] 10.222.0.81 () {56 vars in 1122 bytes} [Mon Sep 19 11:11:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:21.715933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c07017-b624-4642-9dfa-557770502e4e None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:21.715933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c07017-b624-4642-9dfa-557770502e4e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:21.715933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c07017-b624-4642-9dfa-557770502e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:21.732947 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-17c07017-b624-4642-9dfa-557770502e4e None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:21.734188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-17c07017-b624-4642-9dfa-557770502e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:21.793063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:21.809264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 473/929] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:21.821318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c07017-b624-4642-9dfa-557770502e4e None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:21.834379 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c989d-1fbc-4c73-a2af-53af8c96fa5f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:21.873236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688d85b-b44a-4d15-9eaa-71d1106e598a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:21.874163 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688d85b-b44a-4d15-9eaa-71d1106e598a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:21.874641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688d85b-b44a-4d15-9eaa-71d1106e598a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:21.875579 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688d85b-b44a-4d15-9eaa-71d1106e598a None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:21.876403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1688d85b-b44a-4d15-9eaa-71d1106e598a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:21.877285 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1688d85b-b44a-4d15-9eaa-71d1106e598a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:21.879116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1688d85b-b44a-4d15-9eaa-71d1106e598a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:21.886707 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 457/930] 10.222.0.81 () {56 vars in 1126 bytes} [Mon Sep 19 11:11:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:21.891937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688d85b-b44a-4d15-9eaa-71d1106e598a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:21.907823 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:21.939251 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c07017-b624-4642-9dfa-557770502e4e None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:21.978788 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e64fd7-2e84-4e9b-8ccc-14281e551c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:21.979849 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e64fd7-2e84-4e9b-8ccc-14281e551c7e None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:21.980292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e64fd7-2e84-4e9b-8ccc-14281e551c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:21.980957 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e64fd7-2e84-4e9b-8ccc-14281e551c7e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:21.982632 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e64fd7-2e84-4e9b-8ccc-14281e551c7e None admin] RBAC: Adding query filter params (interface=public, service_id=27bf2ab3ad0f48a68d4c2c5e587b506d, region_id=RegionOne) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:21.983088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e64fd7-2e84-4e9b-8ccc-14281e551c7e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:21.984608 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e64fd7-2e84-4e9b-8ccc-14281e551c7e None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:21.999238 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 474/931] 10.222.0.81 () {56 vars in 1251 bytes} [Mon Sep 19 11:11:21 2022] GET /identity/v3/endpoints?service_id=27bf2ab3ad0f48a68d4c2c5e587b506d&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:21.999927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e64fd7-2e84-4e9b-8ccc-14281e551c7e None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:22.020536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688d85b-b44a-4d15-9eaa-71d1106e598a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:22.050241 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:22.056822 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8ec81-7e39-4556-b254-cd4d2bee0e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:22.057690 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8ec81-7e39-4556-b254-cd4d2bee0e56 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:22.058166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8ec81-7e39-4556-b254-cd4d2bee0e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:22.058643 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8ec81-7e39-4556-b254-cd4d2bee0e56 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:22.059504 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da8ec81-7e39-4556-b254-cd4d2bee0e56 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:22.061097 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da8ec81-7e39-4556-b254-cd4d2bee0e56 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:22.087805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 458/932] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:11:21 2022] POST /identity/v3/endpoints => generated 347 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:11:22.087805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8ec81-7e39-4556-b254-cd4d2bee0e56 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:22.115329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e64fd7-2e84-4e9b-8ccc-14281e551c7e None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:22.160425 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:22.166237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8376ab3-d0c8-4d40-a6c8-4181b4a726fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:22.166959 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8376ab3-d0c8-4d40-a6c8-4181b4a726fc None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:22.167451 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8376ab3-d0c8-4d40-a6c8-4181b4a726fc None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:22.168063 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8376ab3-d0c8-4d40-a6c8-4181b4a726fc None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:22.168773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8376ab3-d0c8-4d40-a6c8-4181b4a726fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:22.170268 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8376ab3-d0c8-4d40-a6c8-4181b4a726fc None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:22.177561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 475/933] 10.222.0.81 () {56 vars in 1087 bytes} [Mon Sep 19 11:11:21 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:11:22.209161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8ec81-7e39-4556-b254-cd4d2bee0e56 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:22.249371 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c29c9e-5024-4172-a9f2-f5232301257c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:22.250282 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c29c9e-5024-4172-a9f2-f5232301257c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:22.250729 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c29c9e-5024-4172-a9f2-f5232301257c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:22.251168 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c29c9e-5024-4172-a9f2-f5232301257c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:22.264901 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c29c9e-5024-4172-a9f2-f5232301257c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:11:22.271409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-b7c29c9e-5024-4172-a9f2-f5232301257c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:11:22.271409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:22.271409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:22.271409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:22.271409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:22.271409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:22.271409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:22.271409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:22.271409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:22.271409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:22.271409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:22.271409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:22.271409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:11:22.271409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:11:22.271409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:11:22.271409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:11:22.273827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:11:22.273827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:11:22.273827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:11:22.273827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:11:22.273827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:22.273827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:22.273827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:22.273827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:22.273827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:22.273827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:22.273827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:22.273827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:22.273827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:22.273827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:22.273827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:22.273827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:22.274919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:22.274919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:22.274919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:22.274919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:22.274919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:11:22.274919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:11:22.274919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:11:22.274919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:11:22.274919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:11:22.274919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:11:22.274919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:11:22.274919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:22.277101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 459/934] 10.222.0.81 () {56 vars in 1120 bytes} [Mon Sep 19 11:11:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:22.284590 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8376ab3-d0c8-4d40-a6c8-4181b4a726fc None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:22.396245 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8376ab3-d0c8-4d40-a6c8-4181b4a726fc None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:22.432839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c42e3c-416b-4b08-9c2f-ccbcfc23160b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:22.433444 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c42e3c-416b-4b08-9c2f-ccbcfc23160b None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:22.433720 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c42e3c-416b-4b08-9c2f-ccbcfc23160b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:22.433989 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c42e3c-416b-4b08-9c2f-ccbcfc23160b None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:22.434646 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c42e3c-416b-4b08-9c2f-ccbcfc23160b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:22.434946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c42e3c-416b-4b08-9c2f-ccbcfc23160b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:22.436326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c42e3c-416b-4b08-9c2f-ccbcfc23160b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:22.449780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 476/935] 10.222.0.81 () {56 vars in 1115 bytes} [Mon Sep 19 11:11:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:22.451111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c29c9e-5024-4172-a9f2-f5232301257c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:22.505705 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:22.566304 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c29c9e-5024-4172-a9f2-f5232301257c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:22.603449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85bc678-b645-46e3-8776-a3e1879372e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:22.604106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85bc678-b645-46e3-8776-a3e1879372e4 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:22.604379 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85bc678-b645-46e3-8776-a3e1879372e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:22.604656 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85bc678-b645-46e3-8776-a3e1879372e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:22.615347 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85bc678-b645-46e3-8776-a3e1879372e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:22.617339 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85bc678-b645-46e3-8776-a3e1879372e4 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:22.631413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-a85bc678-b645-46e3-8776-a3e1879372e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:22.631413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:22.631413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:22.631413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:22.631413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:22.631413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:22.631413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:22.631413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:22.631413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:22.631413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:22.631413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:22.631413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:22.631413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:22.631413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:22.631413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:22.631413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:22.633077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:22.633077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:22.633077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:22.633077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:22.633077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:22.633077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:22.633077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:22.633077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:22.633077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:22.633077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:22.633077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:22.633077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:22.633077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:22.633077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:22.633077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:22.633077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:22.633077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:22.633994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:22.633994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:22.633994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:22.633994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 460/936] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:22.639638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c42e3c-416b-4b08-9c2f-ccbcfc23160b None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:22.719440 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:22.766112 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c42e3c-416b-4b08-9c2f-ccbcfc23160b None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:22.806725 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b86356-4d5e-4d15-aaec-cd79811f1f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:22.807337 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b86356-4d5e-4d15-aaec-cd79811f1f52 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:22.807613 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b86356-4d5e-4d15-aaec-cd79811f1f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:22.807922 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b86356-4d5e-4d15-aaec-cd79811f1f52 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:22.808558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b86356-4d5e-4d15-aaec-cd79811f1f52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:22.808861 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b86356-4d5e-4d15-aaec-cd79811f1f52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:22.810215 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b86356-4d5e-4d15-aaec-cd79811f1f52 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:22.827511 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 477/937] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:22.829156 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85bc678-b645-46e3-8776-a3e1879372e4 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:22.929959 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:22.933069 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85bc678-b645-46e3-8776-a3e1879372e4 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:22.968922 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f936cb-16ba-4fa5-ae88-c18402bdde87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:22.969547 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f936cb-16ba-4fa5-ae88-c18402bdde87 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:22.969821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f936cb-16ba-4fa5-ae88-c18402bdde87 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:22.970309 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f936cb-16ba-4fa5-ae88-c18402bdde87 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:22.978932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f936cb-16ba-4fa5-ae88-c18402bdde87 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:22.980410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f936cb-16ba-4fa5-ae88-c18402bdde87 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:22.992215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-52f936cb-16ba-4fa5-ae88-c18402bdde87 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:11:22.992215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:22.992215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:22.992215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:22.992215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:22.992215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:22.992215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:22.992215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:22.992215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:22.992215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:22.992215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:22.992215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:22.992215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:11:22.992215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:11:22.992215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:11:22.992215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:11:22.993213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:22.993213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:22.993213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:11:22.993213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:22.993213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:11:22.993213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:22.993213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:22.993213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:22.993213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:22.993213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:22.993213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:22.993213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:22.993213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:22.993213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:22.993213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:22.993213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:22.993213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:22.993213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:22.994169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:22.994169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:22.994169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:11:22.994169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:11:22.994169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:11:22.994169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:11:22.994169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:11:22.994169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:11:22.994169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:11:22.994169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:22.994169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 461/938] 10.222.0.81 () {56 vars in 1117 bytes} [Mon Sep 19 11:11:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:23.000927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b86356-4d5e-4d15-aaec-cd79811f1f52 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:23.115427 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b86356-4d5e-4d15-aaec-cd79811f1f52 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:23.134177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:23.152286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e2db2-0ed5-49d8-bbd2-7407592e1f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:23.152886 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e2db2-0ed5-49d8-bbd2-7407592e1f50 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:23.153159 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e2db2-0ed5-49d8-bbd2-7407592e1f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:23.153423 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e2db2-0ed5-49d8-bbd2-7407592e1f50 None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:23.154142 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e2db2-0ed5-49d8-bbd2-7407592e1f50 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:23.154442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e2db2-0ed5-49d8-bbd2-7407592e1f50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:23.155938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e2db2-0ed5-49d8-bbd2-7407592e1f50 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:23.162287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501392a0-52e3-4be2-8632-32b29259ed48 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:23.162891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501392a0-52e3-4be2-8632-32b29259ed48 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:23.163368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501392a0-52e3-4be2-8632-32b29259ed48 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:23.166054 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 462/939] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:23.172121 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de586b1-335e-4887-ae4b-a820ab14730b None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:23.172833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de586b1-335e-4887-ae4b-a820ab14730b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:23.173334 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de586b1-335e-4887-ae4b-a820ab14730b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:23.181210 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 478/940] 10.222.0.81 () {56 vars in 1149 bytes} [Mon Sep 19 11:11:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:23.182549 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e2db2-0ed5-49d8-bbd2-7407592e1f50 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:23.193197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6de586b1-335e-4887-ae4b-a820ab14730b None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:23.195309 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6de586b1-335e-4887-ae4b-a820ab14730b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:23.283794 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 463/941] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:11:23.303188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e2db2-0ed5-49d8-bbd2-7407592e1f50 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:23.339576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16325a12-b663-4c8b-b36b-5f8cb9944689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:23.340242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16325a12-b663-4c8b-b36b-5f8cb9944689 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:23.340521 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16325a12-b663-4c8b-b36b-5f8cb9944689 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:23.340783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16325a12-b663-4c8b-b36b-5f8cb9944689 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:23.351146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16325a12-b663-4c8b-b36b-5f8cb9944689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:23.352589 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16325a12-b663-4c8b-b36b-5f8cb9944689 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:23.367970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-16325a12-b663-4c8b-b36b-5f8cb9944689 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:23.367970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:23.367970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:23.367970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:23.367970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:23.367970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:23.367970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:23.367970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:23.367970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:23.367970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:23.367970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:23.367970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:23.367970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:23.367970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:23.367970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:23.367970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:23.368920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:23.368920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:23.368920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:23.368920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:23.368920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:23.368920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:23.368920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:23.368920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:23.368920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:23.368920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:23.368920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:23.368920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:23.368920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:23.368920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:23.368920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:23.368920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:23.368920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:23.369832 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:23.369832 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:23.369832 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:23.373553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 479/942] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:23.374926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de586b1-335e-4887-ae4b-a820ab14730b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:23.472499 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:23.482197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de586b1-335e-4887-ae4b-a820ab14730b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:23.518281 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa2646-a10f-497c-94ec-39f3b0747d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:23.518884 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa2646-a10f-497c-94ec-39f3b0747d2e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:23.519159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa2646-a10f-497c-94ec-39f3b0747d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:23.519427 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa2646-a10f-497c-94ec-39f3b0747d2e None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:23.520161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa2646-a10f-497c-94ec-39f3b0747d2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:23.520460 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa2646-a10f-497c-94ec-39f3b0747d2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:23.521820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa2646-a10f-497c-94ec-39f3b0747d2e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:23.534326 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 464/943] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:23.539734 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16325a12-b663-4c8b-b36b-5f8cb9944689 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:23.577609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:23.595474 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa2646-a10f-497c-94ec-39f3b0747d2e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:23.653799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16325a12-b663-4c8b-b36b-5f8cb9944689 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:23.688821 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d3b74-ba2d-4631-8c38-ec200d2fb465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:23.689444 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d3b74-ba2d-4631-8c38-ec200d2fb465 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:23.689721 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d3b74-ba2d-4631-8c38-ec200d2fb465 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:23.689993 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d3b74-ba2d-4631-8c38-ec200d2fb465 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:23.697451 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d3b74-ba2d-4631-8c38-ec200d2fb465 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:23.698879 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d3b74-ba2d-4631-8c38-ec200d2fb465 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:23.705702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa2646-a10f-497c-94ec-39f3b0747d2e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:23.711059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-383d3b74-ba2d-4631-8c38-ec200d2fb465 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:23.711059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:23.711059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:23.711059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:23.711059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:23.711059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:23.711059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:23.711059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:23.711059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:23.711059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:23.711059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:23.711059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:23.711059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:11:23.711059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:11:23.711059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:11:23.711964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:11:23.711964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:23.711964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:23.711964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:23.711964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:23.711964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:23.711964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:23.711964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:23.711964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:23.711964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:23.711964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:23.711964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:23.711964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:23.711964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:23.711964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:23.711964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:23.711964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:23.712862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:11:23.712862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:11:23.712862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:11:23.712862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:11:23.712862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:11:23.712862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:11:23.712862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:23.712862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:23.712862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 480/944] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:11:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:23.720671 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d3b74-ba2d-4631-8c38-ec200d2fb465 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:23.744858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d7c18-e10d-4dea-b344-50d12342019f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:23.745638 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d7c18-e10d-4dea-b344-50d12342019f None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:23.746007 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d7c18-e10d-4dea-b344-50d12342019f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:23.746405 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d7c18-e10d-4dea-b344-50d12342019f None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:23.747146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216d7c18-e10d-4dea-b344-50d12342019f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:23.748821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216d7c18-e10d-4dea-b344-50d12342019f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:23.751906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: WARNING keystone.server.flask.common [None req-216d7c18-e10d-4dea-b344-50d12342019f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 19 11:11:23.772265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.sql.core [None req-216d7c18-e10d-4dea-b344-50d12342019f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 19 11:11:23.772265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 19 11:11:23.772265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [parameters: {'id': '8329bb052d244a64bd609ff1ce210f8c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 19 11:11:23.772265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81460) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 19 11:11:23.790811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-216d7c18-e10d-4dea-b344-50d12342019f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8329bb052d244a64bd609ff1ce210f8c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8329bb052d244a64bd609ff1ce210f8c. Sep 19 11:11:23.790811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:23.790811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 11:11:23.790811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 11:11:23.790811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 11:11:23.790811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 11:11:23.790811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 11:11:23.790811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 11:11:23.790811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 11:11:23.790811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application conn.query(q) Sep 19 11:11:23.790811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 11:11:23.790811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 11:11:23.790811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 11:11:23.791816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application result.read() Sep 19 11:11:23.791816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 11:11:23.791816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 11:11:23.791816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 11:11:23.791816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 11:11:23.791816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 11:11:23.791816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 11:11:23.791816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 11:11:23.791816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 11:11:23.791816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 19 11:11:23.791816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:23.791816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 19 11:11:23.791816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:23.791816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:23.791816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 19 11:11:23.791816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 19 11:11:23.791816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 19 11:11:23.791816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 19 11:11:23.792973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 11:11:23.792973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application next(self.gen) Sep 19 11:11:23.792973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 19 11:11:23.792973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application yield resource Sep 19 11:11:23.792973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 11:11:23.792973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application next(self.gen) Sep 19 11:11:23.792973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 19 11:11:23.792973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self.session.rollback() Sep 19 11:11:23.792973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 11:11:23.792973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self.force_reraise() Sep 19 11:11:23.792973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 11:11:23.792973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise self.value Sep 19 11:11:23.792973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 19 11:11:23.792973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 19 11:11:23.792973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 19 11:11:23.792973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application session.commit() Sep 19 11:11:23.792973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 19 11:11:23.794133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 19 11:11:23.794133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 19 11:11:23.794133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._prepare_impl() Sep 19 11:11:23.794133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 19 11:11:23.794133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self.session.flush() Sep 19 11:11:23.794133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 19 11:11:23.794133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._flush(objects) Sep 19 11:11:23.794133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 19 11:11:23.794133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 19 11:11:23.794133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 19 11:11:23.794133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application compat.raise_( Sep 19 11:11:23.794133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 11:11:23.794133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception Sep 19 11:11:23.794133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 19 11:11:23.794133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application flush_context.execute() Sep 19 11:11:23.794133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 19 11:11:23.794133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rec.execute(self) Sep 19 11:11:23.795786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 19 11:11:23.795786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 19 11:11:23.795786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 19 11:11:23.795786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 19 11:11:23.795786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 19 11:11:23.795786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 19 11:11:23.795786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 19 11:11:23.795786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 19 11:11:23.795786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 19 11:11:23.795786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 19 11:11:23.795786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 19 11:11:23.795786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 19 11:11:23.795786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 19 11:11:23.795786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 19 11:11:23.795786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 19 11:11:23.797283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 19 11:11:23.797283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 11:11:23.797283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception Sep 19 11:11:23.797283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 11:11:23.797283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 11:11:23.797283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 11:11:23.797283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 11:11:23.797283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 11:11:23.797283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 11:11:23.797283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 11:11:23.797283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application conn.query(q) Sep 19 11:11:23.797283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 11:11:23.797283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 11:11:23.797283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 11:11:23.797283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application result.read() Sep 19 11:11:23.797283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 11:11:23.797283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 11:11:23.798807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 11:11:23.798807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 11:11:23.798807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 11:11:23.798807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 11:11:23.798807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 11:11:23.798807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 11:11:23.798807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 19 11:11:23.798807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 19 11:11:23.798807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [parameters: {'id': '8329bb052d244a64bd609ff1ce210f8c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 19 11:11:23.798807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 19 11:11:23.798807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:23.798807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:23.798807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:23.798807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:23.798807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 19 11:11:23.800215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 19 11:11:23.800215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 19 11:11:23.800215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 19 11:11:23.800215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 19 11:11:23.800215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:23.800215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:23.800215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:23.800215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:23.800215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:23.800215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:23.800215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:23.800215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:23.800215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:23.800215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:23.800215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:23.800215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:23.800215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:23.802403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:23.802403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 19 11:11:23.802403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 19 11:11:23.802403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:23.802403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:23.802403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 19 11:11:23.802403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 19 11:11:23.802403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8329bb052d244a64bd609ff1ce210f8c. Sep 19 11:11:23.802403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:23.802403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 465/945] 10.222.0.81 () {60 vars in 1140 bytes} [Mon Sep 19 11:11:23 2022] POST /identity/v3/projects => generated 262 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:23.804364 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d7c18-e10d-4dea-b344-50d12342019f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:23.815306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:23.842701 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:23.860576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d3b74-ba2d-4631-8c38-ec200d2fb465 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:23.902656 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759034b9-61a9-4159-8a7b-9b72c6347b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:23.903645 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759034b9-61a9-4159-8a7b-9b72c6347b76 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:23.904154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759034b9-61a9-4159-8a7b-9b72c6347b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:23.904725 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759034b9-61a9-4159-8a7b-9b72c6347b76 None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:23.906037 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759034b9-61a9-4159-8a7b-9b72c6347b76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:23.906519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759034b9-61a9-4159-8a7b-9b72c6347b76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:23.908529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759034b9-61a9-4159-8a7b-9b72c6347b76 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:23.925068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 481/946] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:11:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:23.925592 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d7c18-e10d-4dea-b344-50d12342019f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:23.935669 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759034b9-61a9-4159-8a7b-9b72c6347b76 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:23.972951 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6acb2-b926-4302-b13f-05e7e6ab6e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:23.973582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6acb2-b926-4302-b13f-05e7e6ab6e13 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:23.973859 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6acb2-b926-4302-b13f-05e7e6ab6e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:23.974333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6acb2-b926-4302-b13f-05e7e6ab6e13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:23.983186 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6acb2-b926-4302-b13f-05e7e6ab6e13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:23.985310 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6acb2-b926-4302-b13f-05e7e6ab6e13 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:23.999774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-25b6acb2-b926-4302-b13f-05e7e6ab6e13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:23.999774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:23.999774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:23.999774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:23.999774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:23.999774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:23.999774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:23.999774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:23.999774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:23.999774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:23.999774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:23.999774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:23.999774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:11:23.999774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:11:23.999774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:11:24.000720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:11:24.000720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:24.000720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:24.000720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:24.000720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:24.000720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:24.000720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:24.000720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:24.000720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:24.000720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:24.000720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:24.000720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:24.000720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:24.000720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:24.000720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:24.000720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:24.000720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:24.001597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:11:24.001597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:11:24.001597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:11:24.001597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:11:24.001597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:11:24.001597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:11:24.001597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:24.001597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:24.002026 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 466/947] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:11:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:24.008781 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6acb2-b926-4302-b13f-05e7e6ab6e13 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:24.056525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759034b9-61a9-4159-8a7b-9b72c6347b76 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:24.079248 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:24.097369 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b875a6a-d636-4b1b-8f37-8d50e9a8b886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:24.098207 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b875a6a-d636-4b1b-8f37-8d50e9a8b886 None admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:24.098729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b875a6a-d636-4b1b-8f37-8d50e9a8b886 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:24.099068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b875a6a-d636-4b1b-8f37-8d50e9a8b886 None admin] PATH_INFO: `/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/06d279b488d148428b7a3c35a9786ffe/roles/7689e410cf25472397444b35809be2c7` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:24.102290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b875a6a-d636-4b1b-8f37-8d50e9a8b886 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed9a0b626e3e4b93b391158e981ec586, user_id=06d279b488d148428b7a3c35a9786ffe, role_id=7689e410cf25472397444b35809be2c7)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:24.103944 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b875a6a-d636-4b1b-8f37-8d50e9a8b886 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:24.122181 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 482/948] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:11:23 2022] PUT /identity/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/06d279b488d148428b7a3c35a9786ffe/roles/7689e410cf25472397444b35809be2c7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:11:24.122628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6acb2-b926-4302-b13f-05e7e6ab6e13 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:24.161963 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687af8a-f8e2-4c98-80cc-e54af990697b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:24.162500 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687af8a-f8e2-4c98-80cc-e54af990697b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:24.162821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687af8a-f8e2-4c98-80cc-e54af990697b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:24.163093 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687af8a-f8e2-4c98-80cc-e54af990697b None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:24.163862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687af8a-f8e2-4c98-80cc-e54af990697b None admin] RBAC: Adding query filter params (name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:24.164168 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687af8a-f8e2-4c98-80cc-e54af990697b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:24.165757 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687af8a-f8e2-4c98-80cc-e54af990697b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:24.189597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 467/949] 10.222.0.81 () {56 vars in 1112 bytes} [Mon Sep 19 11:11:24 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:24.469251 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c641f8-2a28-4692-9b37-74233e931a5c None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:24.469914 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c641f8-2a28-4692-9b37-74233e931a5c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:24.470419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c641f8-2a28-4692-9b37-74233e931a5c None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:24.472259 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 483/950] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:24.478520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8012242-b8b1-4a0f-b161-f299d21fa34d None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:24.479411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8012242-b8b1-4a0f-b161-f299d21fa34d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:24.479938 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8012242-b8b1-4a0f-b161-f299d21fa34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:24.497730 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21134002-c49a-4b29-babf-2021c6d650d4 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:24.498071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21134002-c49a-4b29-babf-2021c6d650d4 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:24.498283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21134002-c49a-4b29-babf-2021c6d650d4 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:24.499620 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 484/951] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:24.500711 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e8012242-b8b1-4a0f-b161-f299d21fa34d None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:24.502525 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e8012242-b8b1-4a0f-b161-f299d21fa34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:24.506476 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee82439-f1ae-4c5f-8d3a-720231a397ec None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:24.506765 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee82439-f1ae-4c5f-8d3a-720231a397ec None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:24.507044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee82439-f1ae-4c5f-8d3a-720231a397ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:24.526164 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-2ee82439-f1ae-4c5f-8d3a-720231a397ec None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:24.527489 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2ee82439-f1ae-4c5f-8d3a-720231a397ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:24.529641 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:24.587141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 468/952] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:24.612046 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 485/953] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:11:24.859180 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8012242-b8b1-4a0f-b161-f299d21fa34d None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:24.889920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee82439-f1ae-4c5f-8d3a-720231a397ec None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:24.923370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:24.974999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8012242-b8b1-4a0f-b161-f299d21fa34d None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:25.017648 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8caca86-fa6a-4012-a3a7-3cf2d9cbeabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:25.018390 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8caca86-fa6a-4012-a3a7-3cf2d9cbeabb None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:25.018675 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8caca86-fa6a-4012-a3a7-3cf2d9cbeabb None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:25.018949 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8caca86-fa6a-4012-a3a7-3cf2d9cbeabb None admin] PATH_INFO: `/v3/services/compute` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:25.019797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8caca86-fa6a-4012-a3a7-3cf2d9cbeabb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:25.021123 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8caca86-fa6a-4012-a3a7-3cf2d9cbeabb None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:25.023700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee82439-f1ae-4c5f-8d3a-720231a397ec None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:25.031264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-d8caca86-fa6a-4012-a3a7-3cf2d9cbeabb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:11:25.031264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:25.031264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:25.031264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:25.031264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:25.031264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:25.031264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:25.031264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:25.031264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:11:25.031264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:25.031264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:25.031264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:25.031264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:25.031264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:25.031264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:25.031264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:25.031264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:25.037870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:25.037870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:25.037870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:25.037870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:25.037870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:25.037870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:25.037870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:25.037870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:25.037870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:25.037870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:25.037870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:25.037870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:25.037870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:25.037870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:25.037870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:25.037870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:25.038848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:25.038848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:25.038848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:25.038848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:25.038848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:25.038848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:25.038848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:25.038848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:25.038848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:25.038848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:25.038848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:25.038848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:25.038848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:11:25.038848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:25.038848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 469/954] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:11:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:25.039699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8caca86-fa6a-4012-a3a7-3cf2d9cbeabb None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:25.070683 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce082fd7-940b-48be-975b-8eb7e2972169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:25.071149 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce082fd7-940b-48be-975b-8eb7e2972169 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:25.071436 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce082fd7-940b-48be-975b-8eb7e2972169 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:25.071711 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce082fd7-940b-48be-975b-8eb7e2972169 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:25.072325 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce082fd7-940b-48be-975b-8eb7e2972169 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:25.073685 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce082fd7-940b-48be-975b-8eb7e2972169 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:25.082963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-ce082fd7-940b-48be-975b-8eb7e2972169 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:11:25.082963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:25.082963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:25.082963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:25.082963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:25.082963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:25.082963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:25.082963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:25.082963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:11:25.082963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:25.082963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:25.082963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:25.082963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:25.082963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:25.082963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:25.082963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:25.082963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:25.084094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:25.084094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:25.084094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:25.084094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:25.084094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:25.084094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:25.084094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:25.084094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:25.084094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:25.084094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:25.084094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:25.084094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:25.084094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:25.084094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:25.084094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:25.084094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:25.084994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:25.084994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:25.084994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:25.084994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:25.084994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:25.084994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:25.084994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:25.084994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:25.084994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:25.084994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:25.084994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:25.084994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:25.084994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:11:25.084994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:25.084994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 486/955] 10.222.0.81 () {56 vars in 1126 bytes} [Mon Sep 19 11:11:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:25.090770 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce082fd7-940b-48be-975b-8eb7e2972169 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:25.093525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:25.166353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8caca86-fa6a-4012-a3a7-3cf2d9cbeabb None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:25.190210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:25.206837 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ffa95-ab2e-475f-b05c-cff1fab6961b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:25.207449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ffa95-ab2e-475f-b05c-cff1fab6961b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:25.207732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ffa95-ab2e-475f-b05c-cff1fab6961b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:25.208045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ffa95-ab2e-475f-b05c-cff1fab6961b None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:25.208773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853ffa95-ab2e-475f-b05c-cff1fab6961b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:25.209080 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853ffa95-ab2e-475f-b05c-cff1fab6961b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:25.210512 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853ffa95-ab2e-475f-b05c-cff1fab6961b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:25.220168 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 470/956] 10.222.0.81 () {56 vars in 1112 bytes} [Mon Sep 19 11:11:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:25.221316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ffa95-ab2e-475f-b05c-cff1fab6961b None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:25.236596 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce082fd7-940b-48be-975b-8eb7e2972169 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:25.281698 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5fb817-9502-4bca-9611-9e2ebda95988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:25.282687 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fb817-9502-4bca-9611-9e2ebda95988 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:25.283178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fb817-9502-4bca-9611-9e2ebda95988 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:25.283666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fb817-9502-4bca-9611-9e2ebda95988 None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:25.285229 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5fb817-9502-4bca-9611-9e2ebda95988 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:25.285741 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5fb817-9502-4bca-9611-9e2ebda95988 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:25.287689 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5fb817-9502-4bca-9611-9e2ebda95988 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:25.295591 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 487/957] 10.222.0.81 () {56 vars in 1122 bytes} [Mon Sep 19 11:11:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:25.301699 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5fb817-9502-4bca-9611-9e2ebda95988 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:25.341604 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ffa95-ab2e-475f-b05c-cff1fab6961b None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:25.377767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cec7d9-719d-48e3-a658-2439587fe739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:25.378368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cec7d9-719d-48e3-a658-2439587fe739 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:25.378646 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cec7d9-719d-48e3-a658-2439587fe739 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:25.378917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cec7d9-719d-48e3-a658-2439587fe739 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:25.379601 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cec7d9-719d-48e3-a658-2439587fe739 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:25.379861 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cec7d9-719d-48e3-a658-2439587fe739 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:25.381228 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cec7d9-719d-48e3-a658-2439587fe739 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:25.389285 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:25.392050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 471/958] 10.222.0.81 () {56 vars in 1112 bytes} [Mon Sep 19 11:11:25 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:25.433552 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5fb817-9502-4bca-9611-9e2ebda95988 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:25.473754 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74c091-b0ce-4f5b-91f1-9f7e627b1818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:25.474781 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74c091-b0ce-4f5b-91f1-9f7e627b1818 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:25.475271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74c091-b0ce-4f5b-91f1-9f7e627b1818 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:25.475750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74c091-b0ce-4f5b-91f1-9f7e627b1818 None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:25.477236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74c091-b0ce-4f5b-91f1-9f7e627b1818 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:25.477756 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74c091-b0ce-4f5b-91f1-9f7e627b1818 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:25.479401 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74c091-b0ce-4f5b-91f1-9f7e627b1818 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:25.485946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 488/959] 10.222.0.81 () {56 vars in 1122 bytes} [Mon Sep 19 11:11:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:25.492180 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cec7d9-719d-48e3-a658-2439587fe739 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:25.549284 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:25.606702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cec7d9-719d-48e3-a658-2439587fe739 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:25.642682 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd959f-5771-43b9-bc5f-84c0d56a3c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:25.643666 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd959f-5771-43b9-bc5f-84c0d56a3c73 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:25.644199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd959f-5771-43b9-bc5f-84c0d56a3c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:25.644682 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd959f-5771-43b9-bc5f-84c0d56a3c73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:25.645615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd959f-5771-43b9-bc5f-84c0d56a3c73 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:25.647213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd959f-5771-43b9-bc5f-84c0d56a3c73 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:25.670798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 472/960] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:11:25 2022] POST /identity/v3/endpoints => generated 349 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:11:26.358015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3604dd-471d-4ad5-816b-ea5416ad662e None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:26.358587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3604dd-471d-4ad5-816b-ea5416ad662e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:26.358587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3604dd-471d-4ad5-816b-ea5416ad662e None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:26.360231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 489/961] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:26.365561 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17f5d7-bd13-41df-b349-82fc0b0d9f46 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:26.365844 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17f5d7-bd13-41df-b349-82fc0b0d9f46 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:26.366124 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17f5d7-bd13-41df-b349-82fc0b0d9f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:26.386783 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-1e17f5d7-bd13-41df-b349-82fc0b0d9f46 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:26.388068 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1e17f5d7-bd13-41df-b349-82fc0b0d9f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:26.469712 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 473/962] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:26.497764 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e541b04-be50-46de-bece-ad33ecc1e7cd None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:26.498065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e541b04-be50-46de-bece-ad33ecc1e7cd None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:26.498308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e541b04-be50-46de-bece-ad33ecc1e7cd None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:26.499894 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 490/963] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:26.505088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8419be-4b83-40d1-ad41-d2ad48d50f9e None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:26.505546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8419be-4b83-40d1-ad41-d2ad48d50f9e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:26.505639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8419be-4b83-40d1-ad41-d2ad48d50f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:26.524811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-0e8419be-4b83-40d1-ad41-d2ad48d50f9e None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:26.526010 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0e8419be-4b83-40d1-ad41-d2ad48d50f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:26.550886 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:26.598881 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 474/964] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:26.760951 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e541b04-be50-46de-bece-ad33ecc1e7cd None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:26.872753 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e541b04-be50-46de-bece-ad33ecc1e7cd None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:26.878856 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8419be-4b83-40d1-ad41-d2ad48d50f9e None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:26.908028 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:26.916811 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ce856-1b0f-48ac-8db5-d7c8d513fb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:26.917769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ce856-1b0f-48ac-8db5-d7c8d513fb87 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:26.918050 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ce856-1b0f-48ac-8db5-d7c8d513fb87 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:26.918330 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ce856-1b0f-48ac-8db5-d7c8d513fb87 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:26.938228 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ce856-1b0f-48ac-8db5-d7c8d513fb87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:11:26.943975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-977ce856-1b0f-48ac-8db5-d7c8d513fb87 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:11:26.943975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:26.943975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:26.943975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:26.943975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:26.943975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:26.943975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:26.943975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:26.943975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:26.943975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:26.943975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:26.943975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:26.943975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:11:26.943975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:11:26.943975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:11:26.943975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:11:26.945480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:11:26.945480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:11:26.945480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:11:26.945480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:11:26.945480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:26.945480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:26.945480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:26.945480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:26.945480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:26.945480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:26.945480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:26.945480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:26.945480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:26.945480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:26.945480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:26.945480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:26.946338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:26.946338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:26.946338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:26.946338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:26.946338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:11:26.946338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:11:26.946338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:11:26.946338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:11:26.946338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:11:26.946338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:11:26.946338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:11:26.946338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:26.947592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 491/965] 10.222.0.81 () {56 vars in 1120 bytes} [Mon Sep 19 11:11:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:26.956713 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ce856-1b0f-48ac-8db5-d7c8d513fb87 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:26.995311 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8419be-4b83-40d1-ad41-d2ad48d50f9e None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:27.033732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8faa52f-0216-4d6e-a60b-70cdc525ea83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:27.034487 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8faa52f-0216-4d6e-a60b-70cdc525ea83 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:27.034890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8faa52f-0216-4d6e-a60b-70cdc525ea83 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:27.035265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8faa52f-0216-4d6e-a60b-70cdc525ea83 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:27.036190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8faa52f-0216-4d6e-a60b-70cdc525ea83 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:27.037607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8faa52f-0216-4d6e-a60b-70cdc525ea83 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:27.058376 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.sql.core [None req-f8faa52f-0216-4d6e-a60b-70cdc525ea83 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 19 11:11:27.058376 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 19 11:11:27.058376 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [parameters: {'user_id': '6edf7f186011403ea75c42ca312b003e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 19 11:11:27.058376 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81460) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 19 11:11:27.062411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ce856-1b0f-48ac-8db5-d7c8d513fb87 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:27.067830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-f8faa52f-0216-4d6e-a60b-70cdc525ea83 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 19 11:11:27.067830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:27.067830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 11:11:27.067830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 11:11:27.067830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 11:11:27.067830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 11:11:27.067830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 11:11:27.067830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 11:11:27.067830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 11:11:27.067830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application conn.query(q) Sep 19 11:11:27.067830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 11:11:27.067830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 11:11:27.067830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 11:11:27.067830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application result.read() Sep 19 11:11:27.067830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 11:11:27.068761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 11:11:27.068761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 11:11:27.068761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 11:11:27.068761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 11:11:27.068761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 11:11:27.068761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 11:11:27.068761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 11:11:27.068761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 19 11:11:27.068761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:27.068761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 19 11:11:27.068761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:27.068761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:27.068761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 19 11:11:27.068761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 19 11:11:27.068761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 19 11:11:27.068761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 19 11:11:27.068761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 11:11:27.068761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application next(self.gen) Sep 19 11:11:27.069851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 19 11:11:27.069851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application yield resource Sep 19 11:11:27.069851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 11:11:27.069851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application next(self.gen) Sep 19 11:11:27.069851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 19 11:11:27.069851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self.session.rollback() Sep 19 11:11:27.069851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 11:11:27.069851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self.force_reraise() Sep 19 11:11:27.069851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 11:11:27.069851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise self.value Sep 19 11:11:27.069851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 19 11:11:27.069851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 19 11:11:27.069851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 19 11:11:27.069851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application session.commit() Sep 19 11:11:27.069851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 19 11:11:27.069851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 19 11:11:27.069851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 19 11:11:27.070769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._prepare_impl() Sep 19 11:11:27.070769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 19 11:11:27.070769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self.session.flush() Sep 19 11:11:27.070769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 19 11:11:27.070769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._flush(objects) Sep 19 11:11:27.070769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 19 11:11:27.070769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 19 11:11:27.070769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 19 11:11:27.070769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application compat.raise_( Sep 19 11:11:27.070769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 11:11:27.070769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception Sep 19 11:11:27.070769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 19 11:11:27.070769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application flush_context.execute() Sep 19 11:11:27.070769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 19 11:11:27.070769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rec.execute(self) Sep 19 11:11:27.070769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 19 11:11:27.070769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 19 11:11:27.071715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 19 11:11:27.071715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 19 11:11:27.071715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 19 11:11:27.071715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 19 11:11:27.071715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 19 11:11:27.071715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 19 11:11:27.071715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 19 11:11:27.071715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 19 11:11:27.071715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 19 11:11:27.071715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 19 11:11:27.071715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 19 11:11:27.071715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 19 11:11:27.071715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 19 11:11:27.071715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 19 11:11:27.071715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 11:11:27.071715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception Sep 19 11:11:27.074618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 11:11:27.074618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 11:11:27.074618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 11:11:27.074618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 11:11:27.074618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 11:11:27.074618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 11:11:27.074618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 11:11:27.074618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application conn.query(q) Sep 19 11:11:27.074618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 11:11:27.074618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 11:11:27.074618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 11:11:27.074618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application result.read() Sep 19 11:11:27.074618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 11:11:27.074618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 11:11:27.074618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 11:11:27.074618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 11:11:27.076834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 11:11:27.076834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 11:11:27.076834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 11:11:27.076834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 11:11:27.076834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 19 11:11:27.076834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 19 11:11:27.076834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [parameters: {'user_id': '6edf7f186011403ea75c42ca312b003e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 19 11:11:27.076834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 19 11:11:27.076834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:27.076834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:27.076834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:27.076834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:27.076834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:27.076834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:27.076834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:27.079031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:27.079031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:27.079031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:27.079031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:27.079031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:27.079031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:27.079031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:27.079031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 19 11:11:27.079031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 19 11:11:27.079031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:27.079031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:27.079031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:11:27.079031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:27.079031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:11:27.079031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:27.079031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 19 11:11:27.079031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 19 11:11:27.082116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 19 11:11:27.082116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 19 11:11:27.082116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 19 11:11:27.082116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 19 11:11:27.082116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 19 11:11:27.082116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:27.082116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 475/966] 10.222.0.81 () {60 vars in 1131 bytes} [Mon Sep 19 11:11:26 2022] POST /identity/v3/users => generated 160 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:27.082116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8faa52f-0216-4d6e-a60b-70cdc525ea83 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:27.109774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:27.117023 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d99e2-9f35-438b-877e-d703a9e84139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:27.117888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d99e2-9f35-438b-877e-d703a9e84139 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:27.118504 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d99e2-9f35-438b-877e-d703a9e84139 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:27.119045 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d99e2-9f35-438b-877e-d703a9e84139 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:27.120083 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d99e2-9f35-438b-877e-d703a9e84139 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:27.120605 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d99e2-9f35-438b-877e-d703a9e84139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:27.122326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d99e2-9f35-438b-877e-d703a9e84139 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:27.132267 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 492/967] 10.222.0.81 () {56 vars in 1115 bytes} [Mon Sep 19 11:11:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:27.141255 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d99e2-9f35-438b-877e-d703a9e84139 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:27.204689 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:27.230184 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8faa52f-0216-4d6e-a60b-70cdc525ea83 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:27.266691 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d99e2-9f35-438b-877e-d703a9e84139 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:27.271313 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168145b-30ad-43f5-b4ad-5e8fe50258d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:27.272115 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168145b-30ad-43f5-b4ad-5e8fe50258d5 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:27.272471 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168145b-30ad-43f5-b4ad-5e8fe50258d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:27.272930 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168145b-30ad-43f5-b4ad-5e8fe50258d5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:27.282353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7168145b-30ad-43f5-b4ad-5e8fe50258d5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:27.284347 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7168145b-30ad-43f5-b4ad-5e8fe50258d5 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:27.296668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-7168145b-30ad-43f5-b4ad-5e8fe50258d5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:11:27.296668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:27.296668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:27.296668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:27.296668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:27.296668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:27.296668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:27.296668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:27.296668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:27.296668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:27.296668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:27.296668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:27.296668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:11:27.296668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:11:27.296668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:11:27.296668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:11:27.297658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:27.297658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:27.297658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:11:27.297658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:27.297658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:11:27.297658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:27.297658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:27.297658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:27.297658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:27.297658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:27.297658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:27.297658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:27.297658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:27.297658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:27.297658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:27.297658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:27.297658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:27.297658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:27.298641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:27.298641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:27.298641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:11:27.298641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:11:27.298641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:11:27.298641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:11:27.298641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:11:27.298641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:11:27.298641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 11:11:27.298641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:27.302816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 476/968] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:11:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:27.304372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168145b-30ad-43f5-b4ad-5e8fe50258d5 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:27.310379 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8c621-a6b5-4896-bdda-cf69812ce768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:27.311267 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8c621-a6b5-4896-bdda-cf69812ce768 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:27.311725 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8c621-a6b5-4896-bdda-cf69812ce768 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:27.312248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8c621-a6b5-4896-bdda-cf69812ce768 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:27.323573 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8c621-a6b5-4896-bdda-cf69812ce768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:27.325355 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8c621-a6b5-4896-bdda-cf69812ce768 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:27.340712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-9ff8c621-a6b5-4896-bdda-cf69812ce768 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:27.340712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:27.340712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:27.340712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:27.340712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:27.340712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:27.340712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:27.340712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:27.340712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:27.340712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:27.340712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:27.340712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:27.340712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:27.340712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:27.340712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:27.340712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:27.341664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:27.341664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:27.341664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:27.341664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:27.341664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:27.341664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:27.341664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:27.341664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:27.341664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:27.341664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:27.341664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:27.341664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:27.341664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:27.341664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:27.341664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:27.341664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:27.341664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:27.342567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:27.342567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:27.342567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:27.349163 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 493/969] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:27.350668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8c621-a6b5-4896-bdda-cf69812ce768 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:27.395948 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:27.418772 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168145b-30ad-43f5-b4ad-5e8fe50258d5 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:27.460471 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2d2d9-211a-4278-880d-9765ba813c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:27.461038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2d2d9-211a-4278-880d-9765ba813c98 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:27.461324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2d2d9-211a-4278-880d-9765ba813c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:27.461599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2d2d9-211a-4278-880d-9765ba813c98 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:27.462317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2d2d9-211a-4278-880d-9765ba813c98 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:27.462526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8c621-a6b5-4896-bdda-cf69812ce768 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:27.462974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2d2d9-211a-4278-880d-9765ba813c98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:27.465700 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2d2d9-211a-4278-880d-9765ba813c98 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:27.466416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:27.500983 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0716eb-17cb-4c6b-be47-876edcff5f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:27.501743 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0716eb-17cb-4c6b-be47-876edcff5f8a None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:27.502126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0716eb-17cb-4c6b-be47-876edcff5f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:27.502492 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0716eb-17cb-4c6b-be47-876edcff5f8a None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:27.503234 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0716eb-17cb-4c6b-be47-876edcff5f8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:27.503637 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0716eb-17cb-4c6b-be47-876edcff5f8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:27.505176 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0716eb-17cb-4c6b-be47-876edcff5f8a None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:27.516310 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 477/970] 10.222.0.81 () {56 vars in 1101 bytes} [Mon Sep 19 11:11:27 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:27.523449 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 494/971] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:27.525038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2d2d9-211a-4278-880d-9765ba813c98 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:27.557205 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d74b33-943d-4763-b49d-018e78c1b231 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:27.557532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d74b33-943d-4763-b49d-018e78c1b231 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:27.557749 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d74b33-943d-4763-b49d-018e78c1b231 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:27.559416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 495/972] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:27.565455 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19aee0d-51bf-49ed-ada9-8b8b9fd7d03e None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:27.565862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19aee0d-51bf-49ed-ada9-8b8b9fd7d03e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:27.566135 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19aee0d-51bf-49ed-ada9-8b8b9fd7d03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:27.586761 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-b19aee0d-51bf-49ed-ada9-8b8b9fd7d03e None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:27.588275 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b19aee0d-51bf-49ed-ada9-8b8b9fd7d03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:27.643189 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2d2d9-211a-4278-880d-9765ba813c98 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:27.687788 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 496/973] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:27.687995 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd38df-7b1b-4b02-b437-8af05e40d4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:27.687995 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd38df-7b1b-4b02-b437-8af05e40d4c0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:27.687995 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd38df-7b1b-4b02-b437-8af05e40d4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:27.687995 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd38df-7b1b-4b02-b437-8af05e40d4c0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:27.695117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd38df-7b1b-4b02-b437-8af05e40d4c0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:27.696695 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd38df-7b1b-4b02-b437-8af05e40d4c0 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:27.699880 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:27.711119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-cfbd38df-7b1b-4b02-b437-8af05e40d4c0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:11:27.711119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:27.711119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:27.711119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:27.711119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:27.711119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:27.711119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:27.711119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:27.711119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:27.711119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:27.711119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:27.711119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:27.711119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:11:27.711119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:11:27.711119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:11:27.711119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:11:27.712159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:27.712159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:27.712159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:11:27.712159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:27.712159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:11:27.712159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:27.712159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:27.712159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:27.712159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:27.712159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:27.712159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:27.712159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:27.712159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:27.712159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:27.712159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:27.712159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:27.712159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:27.712159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:27.713131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:27.713131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:27.713131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:11:27.713131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:11:27.713131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:11:27.713131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:11:27.713131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:11:27.713131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:11:27.713131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 11:11:27.713131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:27.717123 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 478/974] 10.222.0.81 () {56 vars in 1117 bytes} [Mon Sep 19 11:11:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:27.718568 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19aee0d-51bf-49ed-ada9-8b8b9fd7d03e None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:27.818041 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:27.827020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19aee0d-51bf-49ed-ada9-8b8b9fd7d03e None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:27.867017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7e7b1-7d3d-4da3-9e45-5b50325189a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:27.867636 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7e7b1-7d3d-4da3-9e45-5b50325189a3 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:27.867947 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7e7b1-7d3d-4da3-9e45-5b50325189a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:27.868221 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7e7b1-7d3d-4da3-9e45-5b50325189a3 None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:27.868955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7e7b1-7d3d-4da3-9e45-5b50325189a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:27.869261 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7e7b1-7d3d-4da3-9e45-5b50325189a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:27.870676 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7e7b1-7d3d-4da3-9e45-5b50325189a3 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:27.888838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 497/975] 10.222.0.81 () {56 vars in 1149 bytes} [Mon Sep 19 11:11:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:27.894587 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd38df-7b1b-4b02-b437-8af05e40d4c0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:27.945729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a475e-e46f-4f4a-ad50-cdb0d986b42e None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:27.946038 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a475e-e46f-4f4a-ad50-cdb0d986b42e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:27.946282 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a475e-e46f-4f4a-ad50-cdb0d986b42e None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:27.947736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 498/976] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:11:27.952462 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9bf5c-e990-4a25-8814-a5dfc1ffd273 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:27.952750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9bf5c-e990-4a25-8814-a5dfc1ffd273 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:27.953284 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9bf5c-e990-4a25-8814-a5dfc1ffd273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:27.971145 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-87b9bf5c-e990-4a25-8814-a5dfc1ffd273 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:27.972310 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-87b9bf5c-e990-4a25-8814-a5dfc1ffd273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:27.995723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd38df-7b1b-4b02-b437-8af05e40d4c0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:28.028759 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439f43d-2a78-4230-b3ff-55f44870d84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:28.029362 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439f43d-2a78-4230-b3ff-55f44870d84d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:28.029644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439f43d-2a78-4230-b3ff-55f44870d84d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:28.029915 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439f43d-2a78-4230-b3ff-55f44870d84d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:28.041490 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 499/977] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:11:28.042293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9bf5c-e990-4a25-8814-a5dfc1ffd273 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:28.098732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:28.102582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f439f43d-2a78-4230-b3ff-55f44870d84d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:28.105117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f439f43d-2a78-4230-b3ff-55f44870d84d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:28.127674 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:28.149441 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9bf5c-e990-4a25-8814-a5dfc1ffd273 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:28.186226 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9752089-61de-412f-8af1-8d99e1725fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:28.186826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9752089-61de-412f-8af1-8d99e1725fcc None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:28.187436 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9752089-61de-412f-8af1-8d99e1725fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:28.187436 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9752089-61de-412f-8af1-8d99e1725fcc None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:28.188041 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9752089-61de-412f-8af1-8d99e1725fcc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:28.189397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9752089-61de-412f-8af1-8d99e1725fcc None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:28.205574 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 500/978] 10.222.0.81 () {60 vars in 1141 bytes} [Mon Sep 19 11:11:28 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:11:28.214683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 479/979] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:11:27 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:11:28.216653 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9752089-61de-412f-8af1-8d99e1725fcc None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:28.317899 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439f43d-2a78-4230-b3ff-55f44870d84d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:28.336399 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9752089-61de-412f-8af1-8d99e1725fcc None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:28.376329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca1718-d4b2-4404-b633-7cca42124c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:28.377460 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca1718-d4b2-4404-b633-7cca42124c0c None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:28.377460 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca1718-d4b2-4404-b633-7cca42124c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:28.377634 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca1718-d4b2-4404-b633-7cca42124c0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:28.388288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:28.391214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca1718-d4b2-4404-b633-7cca42124c0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:28.392750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca1718-d4b2-4404-b633-7cca42124c0c None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:28.410532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-83ca1718-d4b2-4404-b633-7cca42124c0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:28.410532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:28.410532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:28.410532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:28.410532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:28.410532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:28.410532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:28.410532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:28.410532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:28.410532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:28.410532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:28.410532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:28.410532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:28.410532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:28.410532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:28.410532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:28.411530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:28.411530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:28.411530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:28.411530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:28.411530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:28.411530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:28.411530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:28.411530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:28.411530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:28.411530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:28.411530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:28.411530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:28.411530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:28.411530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:28.411530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:28.411530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:28.411530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:28.412682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:28.412682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:28.412682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:28.412682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 501/980] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:28.419234 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca1718-d4b2-4404-b633-7cca42124c0c None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:28.429327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:28.432969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439f43d-2a78-4230-b3ff-55f44870d84d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:28.471075 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ad15b-ee6b-40fe-a1d6-b48b05356c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:28.471700 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ad15b-ee6b-40fe-a1d6-b48b05356c94 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:28.472018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ad15b-ee6b-40fe-a1d6-b48b05356c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:28.472294 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ad15b-ee6b-40fe-a1d6-b48b05356c94 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:28.472887 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ad15b-ee6b-40fe-a1d6-b48b05356c94 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:28.474257 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ad15b-ee6b-40fe-a1d6-b48b05356c94 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:28.483823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-c65ad15b-ee6b-40fe-a1d6-b48b05356c94 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:11:28.483823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:28.483823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:28.483823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:28.483823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:28.483823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:28.483823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:28.483823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:28.483823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:11:28.483823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:28.483823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:28.483823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:28.483823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:28.483823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:28.483823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:28.483823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:28.483823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:28.485046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:28.485046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:28.485046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:28.485046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:28.485046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:28.485046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:28.485046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:28.485046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:28.485046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:28.485046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:28.485046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:28.485046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:28.485046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:28.485046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:28.485046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:28.485046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:28.485906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:28.485906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:28.485906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:28.485906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:28.485906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:28.485906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:28.485906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:28.485906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:28.485906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:28.485906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:28.485906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:28.485906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:28.485906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:11:28.485906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:28.485906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 480/981] 10.222.0.81 () {56 vars in 1126 bytes} [Mon Sep 19 11:11:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:28.490741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ad15b-ee6b-40fe-a1d6-b48b05356c94 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:28.529529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca1718-d4b2-4404-b633-7cca42124c0c None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:28.571031 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a7c4b-e62e-422b-ae50-27ca5116a493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:28.571557 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a7c4b-e62e-422b-ae50-27ca5116a493 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:28.571868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a7c4b-e62e-422b-ae50-27ca5116a493 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:28.572145 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a7c4b-e62e-422b-ae50-27ca5116a493 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:28.572791 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a7c4b-e62e-422b-ae50-27ca5116a493 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:28.573095 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a7c4b-e62e-422b-ae50-27ca5116a493 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:28.574452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a7c4b-e62e-422b-ae50-27ca5116a493 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:28.587886 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 502/982] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:28.593209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a7c4b-e62e-422b-ae50-27ca5116a493 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:28.601958 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ad15b-ee6b-40fe-a1d6-b48b05356c94 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:28.638322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad13423-090a-4d81-8149-eb14e7ca7f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:28.638929 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad13423-090a-4d81-8149-eb14e7ca7f26 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:28.639369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad13423-090a-4d81-8149-eb14e7ca7f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:28.639720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad13423-090a-4d81-8149-eb14e7ca7f26 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:28.640455 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad13423-090a-4d81-8149-eb14e7ca7f26 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:28.640765 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad13423-090a-4d81-8149-eb14e7ca7f26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:28.642216 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad13423-090a-4d81-8149-eb14e7ca7f26 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:28.643719 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:28.648671 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 481/983] 10.222.0.81 () {56 vars in 1122 bytes} [Mon Sep 19 11:11:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:28.657613 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad13423-090a-4d81-8149-eb14e7ca7f26 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:28.688977 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:28.717565 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a7c4b-e62e-422b-ae50-27ca5116a493 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:28.758582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f6dd0-9461-435b-b00b-e699d06f5d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:28.759493 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f6dd0-9461-435b-b00b-e699d06f5d0b None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:28.759786 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f6dd0-9461-435b-b00b-e699d06f5d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:28.760092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f6dd0-9461-435b-b00b-e699d06f5d0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:28.766853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad13423-090a-4d81-8149-eb14e7ca7f26 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:28.768840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f6dd0-9461-435b-b00b-e699d06f5d0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:28.771233 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f6dd0-9461-435b-b00b-e699d06f5d0b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:28.785054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-658f6dd0-9461-435b-b00b-e699d06f5d0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:28.785054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:28.785054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:28.785054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:28.785054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:28.785054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:28.785054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:28.785054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:28.785054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:28.785054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:28.785054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:28.785054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:28.785054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:11:28.785054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:11:28.785054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:11:28.785972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:11:28.785972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:28.785972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:28.785972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:28.785972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:28.785972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:28.785972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:28.785972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:28.785972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:28.785972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:28.785972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:28.785972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:28.785972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:28.785972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:28.785972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:28.785972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:28.785972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:28.786906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:11:28.786906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:11:28.786906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:11:28.786906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:11:28.786906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:11:28.786906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:11:28.786906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:28.786906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:28.791358 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 503/984] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:11:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:28.791742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f6dd0-9461-435b-b00b-e699d06f5d0b None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:28.802196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02555e8e-b21c-4ab1-a6e8-2ec3a8b52822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:28.802803 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02555e8e-b21c-4ab1-a6e8-2ec3a8b52822 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:28.803080 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02555e8e-b21c-4ab1-a6e8-2ec3a8b52822 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:28.803347 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02555e8e-b21c-4ab1-a6e8-2ec3a8b52822 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:28.804510 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02555e8e-b21c-4ab1-a6e8-2ec3a8b52822 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:28.804813 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02555e8e-b21c-4ab1-a6e8-2ec3a8b52822 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:28.806181 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02555e8e-b21c-4ab1-a6e8-2ec3a8b52822 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:28.811119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 482/985] 10.222.0.81 () {56 vars in 1122 bytes} [Mon Sep 19 11:11:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:28.816732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02555e8e-b21c-4ab1-a6e8-2ec3a8b52822 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:28.900513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f6dd0-9461-435b-b00b-e699d06f5d0b None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:28.912412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:28.912982 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:28.923134 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02555e8e-b21c-4ab1-a6e8-2ec3a8b52822 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:28.938587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62652c4d-c575-4594-8e18-bd82823cf499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:28.939282 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62652c4d-c575-4594-8e18-bd82823cf499 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:28.939631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62652c4d-c575-4594-8e18-bd82823cf499 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:28.939995 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62652c4d-c575-4594-8e18-bd82823cf499 None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:28.940758 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62652c4d-c575-4594-8e18-bd82823cf499 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:28.941136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62652c4d-c575-4594-8e18-bd82823cf499 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:28.942642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62652c4d-c575-4594-8e18-bd82823cf499 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:28.958106 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 504/986] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:11:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:28.964028 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62652c4d-c575-4594-8e18-bd82823cf499 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:28.966702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2b619-7f7b-4fa9-8d3e-5618d6c7366b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:28.967410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2b619-7f7b-4fa9-8d3e-5618d6c7366b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:28.967783 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2b619-7f7b-4fa9-8d3e-5618d6c7366b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:28.968136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2b619-7f7b-4fa9-8d3e-5618d6c7366b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:28.968927 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e2b619-7f7b-4fa9-8d3e-5618d6c7366b None admin] RBAC: Adding query filter params (interface=admin, service_id=27bf2ab3ad0f48a68d4c2c5e587b506d, region_id=RegionOne) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:28.969306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e2b619-7f7b-4fa9-8d3e-5618d6c7366b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:28.970854 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e2b619-7f7b-4fa9-8d3e-5618d6c7366b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:28.981298 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 483/987] 10.222.0.81 () {56 vars in 1249 bytes} [Mon Sep 19 11:11:28 2022] GET /identity/v3/endpoints?service_id=27bf2ab3ad0f48a68d4c2c5e587b506d&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:28.982495 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2b619-7f7b-4fa9-8d3e-5618d6c7366b None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:29.065663 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62652c4d-c575-4594-8e18-bd82823cf499 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:29.087354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2b619-7f7b-4fa9-8d3e-5618d6c7366b None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:29.101709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0fb0d-f558-44e1-9a7e-709afcd0fada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:29.102392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0fb0d-f558-44e1-9a7e-709afcd0fada None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:29.102754 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0fb0d-f558-44e1-9a7e-709afcd0fada None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:29.103101 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0fb0d-f558-44e1-9a7e-709afcd0fada None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:29.103928 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0fb0d-f558-44e1-9a7e-709afcd0fada None admin] RBAC: Adding query filter params (role.id=7689e410cf25472397444b35809be2c7, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=06d279b488d148428b7a3c35a9786ffe) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:29.104310 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0fb0d-f558-44e1-9a7e-709afcd0fada None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:29.105892 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0fb0d-f558-44e1-9a7e-709afcd0fada None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:29.112676 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 505/988] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:11:28 2022] GET /identity/v3/role_assignments?user.id=06d279b488d148428b7a3c35a9786ffe&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=7689e410cf25472397444b35809be2c7 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:29.131951 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee2da5-9719-47ed-94f3-cc9675faf299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:29.132848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee2da5-9719-47ed-94f3-cc9675faf299 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:29.133281 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee2da5-9719-47ed-94f3-cc9675faf299 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:29.133728 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee2da5-9719-47ed-94f3-cc9675faf299 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:29.134506 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ee2da5-9719-47ed-94f3-cc9675faf299 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:29.136109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ee2da5-9719-47ed-94f3-cc9675faf299 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:29.151956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 484/989] 10.222.0.81 () {56 vars in 1087 bytes} [Mon Sep 19 11:11:28 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:11:30.395314 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03cb83-93e7-4bb3-b550-efd76db12709 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:30.396106 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03cb83-93e7-4bb3-b550-efd76db12709 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:30.396106 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03cb83-93e7-4bb3-b550-efd76db12709 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:30.397250 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 506/990] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:30.403142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cc993-2ea1-4b78-85d0-2216a7790e09 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:30.403542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cc993-2ea1-4b78-85d0-2216a7790e09 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:30.403721 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cc993-2ea1-4b78-85d0-2216a7790e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:30.422370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c11cc993-2ea1-4b78-85d0-2216a7790e09 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:30.423947 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c11cc993-2ea1-4b78-85d0-2216a7790e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:30.438464 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:30.493837 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 485/991] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:30.764873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03cb83-93e7-4bb3-b550-efd76db12709 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:30.800622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:30.869813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03cb83-93e7-4bb3-b550-efd76db12709 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:30.904084 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dce31f-c104-43f6-900d-1845cd46901b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:30.904708 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dce31f-c104-43f6-900d-1845cd46901b None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:30.904985 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dce31f-c104-43f6-900d-1845cd46901b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:30.905256 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dce31f-c104-43f6-900d-1845cd46901b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:30.905830 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dce31f-c104-43f6-900d-1845cd46901b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:30.907172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dce31f-c104-43f6-900d-1845cd46901b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:30.915953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-58dce31f-c104-43f6-900d-1845cd46901b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:11:30.915953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:30.915953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:30.915953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:30.915953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:30.915953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:30.915953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:30.915953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:30.915953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:11:30.915953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:30.915953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:30.915953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:30.915953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:30.915953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:30.915953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:30.915953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:30.915953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:30.916939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:30.916939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:30.916939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:30.916939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:30.916939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:30.916939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:30.916939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:30.916939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:30.916939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:30.916939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:30.916939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:30.916939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:30.916939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:30.916939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:30.916939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:30.916939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:30.917795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:30.917795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:30.917795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:30.917795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:30.917795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:30.917795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:30.917795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:30.917795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:30.917795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:30.917795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:30.917795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:30.917795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:30.917795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:11:30.917795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:30.920083 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 507/992] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:11:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:30.923095 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11cc993-2ea1-4b78-85d0-2216a7790e09 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:31.022925 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11cc993-2ea1-4b78-85d0-2216a7790e09 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:31.056818 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e342d-402d-419d-9cfb-360a7fc003a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:31.057431 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e342d-402d-419d-9cfb-360a7fc003a1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:31.057712 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e342d-402d-419d-9cfb-360a7fc003a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:31.057985 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e342d-402d-419d-9cfb-360a7fc003a1 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:31.058614 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938e342d-402d-419d-9cfb-360a7fc003a1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:31.058918 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938e342d-402d-419d-9cfb-360a7fc003a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:31.060330 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938e342d-402d-419d-9cfb-360a7fc003a1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:31.070406 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 486/993] 10.222.0.81 () {56 vars in 1112 bytes} [Mon Sep 19 11:11:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:31.071593 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dce31f-c104-43f6-900d-1845cd46901b None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:31.169212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:31.174013 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dce31f-c104-43f6-900d-1845cd46901b None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:31.211919 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942d0ba-10a8-4ddb-9c8b-c13dd17ef9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:31.212564 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942d0ba-10a8-4ddb-9c8b-c13dd17ef9ad None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:31.212841 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942d0ba-10a8-4ddb-9c8b-c13dd17ef9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:31.213111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942d0ba-10a8-4ddb-9c8b-c13dd17ef9ad None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:31.213779 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942d0ba-10a8-4ddb-9c8b-c13dd17ef9ad None admin] RBAC: Adding query filter params (type=compute) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:31.214087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942d0ba-10a8-4ddb-9c8b-c13dd17ef9ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:31.215533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942d0ba-10a8-4ddb-9c8b-c13dd17ef9ad None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:31.221609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 508/994] 10.222.0.81 () {56 vars in 1112 bytes} [Mon Sep 19 11:11:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:31.228181 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e342d-402d-419d-9cfb-360a7fc003a1 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:31.245776 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:31.317674 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd31267-b00c-46db-b7a4-13d2f5d829fa None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:31.318078 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd31267-b00c-46db-b7a4-13d2f5d829fa None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:31.318419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd31267-b00c-46db-b7a4-13d2f5d829fa None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:31.320204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 509/995] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:11:31.325211 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6955b1c-fb3d-4504-8758-28b14fd66be9 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:31.325491 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6955b1c-fb3d-4504-8758-28b14fd66be9 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:31.325758 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6955b1c-fb3d-4504-8758-28b14fd66be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:31.343974 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-f6955b1c-fb3d-4504-8758-28b14fd66be9 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:31.345153 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f6955b1c-fb3d-4504-8758-28b14fd66be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:31.345599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e342d-402d-419d-9cfb-360a7fc003a1 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:31.379969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268ea3e-854d-4d4d-a1e3-1962db6e3a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:31.380563 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268ea3e-854d-4d4d-a1e3-1962db6e3a21 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:31.380842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268ea3e-854d-4d4d-a1e3-1962db6e3a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:31.381110 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268ea3e-854d-4d4d-a1e3-1962db6e3a21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:31.382291 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268ea3e-854d-4d4d-a1e3-1962db6e3a21 None admin] RBAC: Adding query filter params (interface=public, service_id=c31530d10fbb4e66bb66eccd31bea28a, region_id=RegionOne) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:31.382597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268ea3e-854d-4d4d-a1e3-1962db6e3a21 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:31.383986 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268ea3e-854d-4d4d-a1e3-1962db6e3a21 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:31.389061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 487/996] 10.222.0.81 () {56 vars in 1251 bytes} [Mon Sep 19 11:11:31 2022] GET /identity/v3/endpoints?service_id=c31530d10fbb4e66bb66eccd31bea28a&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:31.390172 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691bed0-8fc3-4c9f-9d8f-423004d3e52a None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:31.390457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691bed0-8fc3-4c9f-9d8f-423004d3e52a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:31.390726 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691bed0-8fc3-4c9f-9d8f-423004d3e52a None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:31.391942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 488/997] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:31.394361 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691bed0-8fc3-4c9f-9d8f-423004d3e52a None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:31.416970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 510/998] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:11:31.418200 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a129a-e671-4b75-acec-1e80e4f7d629 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:31.418491 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a129a-e671-4b75-acec-1e80e4f7d629 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:31.418924 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a129a-e671-4b75-acec-1e80e4f7d629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:31.437218 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-778a129a-e671-4b75-acec-1e80e4f7d629 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:31.438819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-778a129a-e671-4b75-acec-1e80e4f7d629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:31.480050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:31.503777 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691bed0-8fc3-4c9f-9d8f-423004d3e52a None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:31.527785 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 511/999] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:31.546265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcf26e-d36e-4f4a-a479-9c1f352fce93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:31.546961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcf26e-d36e-4f4a-a479-9c1f352fce93 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:31.547315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcf26e-d36e-4f4a-a479-9c1f352fce93 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:31.547745 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcf26e-d36e-4f4a-a479-9c1f352fce93 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:31.548354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bcf26e-d36e-4f4a-a479-9c1f352fce93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:31.549826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bcf26e-d36e-4f4a-a479-9c1f352fce93 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:31.557036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 489/1000] 10.222.0.81 () {56 vars in 1087 bytes} [Mon Sep 19 11:11:31 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:11:31.691588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a129a-e671-4b75-acec-1e80e4f7d629 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:31.718839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:31.795473 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcf26e-d36e-4f4a-a479-9c1f352fce93 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:31.798975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a129a-e671-4b75-acec-1e80e4f7d629 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:31.837248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9777a49-88c8-4a08-a5b0-57ac04c4431c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:31.837875 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9777a49-88c8-4a08-a5b0-57ac04c4431c None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:31.838151 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9777a49-88c8-4a08-a5b0-57ac04c4431c None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:31.838417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9777a49-88c8-4a08-a5b0-57ac04c4431c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:31.838995 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9777a49-88c8-4a08-a5b0-57ac04c4431c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:31.840384 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9777a49-88c8-4a08-a5b0-57ac04c4431c None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:31.849533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-c9777a49-88c8-4a08-a5b0-57ac04c4431c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:11:31.849533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:31.849533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:31.849533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:31.849533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:31.849533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:31.849533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:31.849533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:31.849533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:11:31.849533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:31.849533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:31.849533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:31.849533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:31.849533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:31.849533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:31.849533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:31.849533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:31.851587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:31.851587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:31.851587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:31.851587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:31.851587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:31.851587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:31.851587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:31.851587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:31.851587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:31.851587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:31.851587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:31.851587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:31.851587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:31.851587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:31.851587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:31.851587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:31.853366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:31.853366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:31.853366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:31.853366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:31.853366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:31.853366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:31.853366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:31.853366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:31.853366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:31.853366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:31.853366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:31.853366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:31.853366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 11:11:31.853366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:31.853366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 512/1001] 10.222.0.81 () {56 vars in 1126 bytes} [Mon Sep 19 11:11:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:31.857211 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9777a49-88c8-4a08-a5b0-57ac04c4431c None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:31.920030 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcf26e-d36e-4f4a-a479-9c1f352fce93 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:31.953390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:31.956516 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3fb3b-d2ba-46d4-9146-1be555b2fc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:31.957140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3fb3b-d2ba-46d4-9146-1be555b2fc35 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:31.957418 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3fb3b-d2ba-46d4-9146-1be555b2fc35 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:31.957689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3fb3b-d2ba-46d4-9146-1be555b2fc35 None admin] PATH_INFO: `/v3/services/image` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:31.958269 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3fb3b-d2ba-46d4-9146-1be555b2fc35 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:31.959638 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3fb3b-d2ba-46d4-9146-1be555b2fc35 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:31.969197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-bfb3fb3b-d2ba-46d4-9146-1be555b2fc35 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:11:31.969197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:31.969197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:31.969197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:31.969197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:31.969197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:31.969197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:31.969197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:31.969197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:11:31.969197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:31.969197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:31.969197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:31.969197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:31.969197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:31.969197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:31.969197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:31.969197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:31.970322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:31.970322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:31.970322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:31.970322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:31.970322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:31.970322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:31.970322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:31.970322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:31.970322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:31.970322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:31.970322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:31.970322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:31.970322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:31.970322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:31.970322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:31.970322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:31.971198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:31.971198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:31.971198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:31.971198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:31.971198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:31.971198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:31.971198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:31.971198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:31.971198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:31.971198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:31.971198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:31.971198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:31.971198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:11:31.971198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:31.971198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 490/1002] 10.222.0.81 () {56 vars in 1105 bytes} [Mon Sep 19 11:11:31 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:31.974844 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9777a49-88c8-4a08-a5b0-57ac04c4431c None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:31.976183 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3fb3b-d2ba-46d4-9146-1be555b2fc35 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:31.985619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:32.011303 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4e553-85fd-4c30-8d21-421adcd79981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:32.011994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4e553-85fd-4c30-8d21-421adcd79981 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:32.012275 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4e553-85fd-4c30-8d21-421adcd79981 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:32.012541 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4e553-85fd-4c30-8d21-421adcd79981 None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:32.013192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4e553-85fd-4c30-8d21-421adcd79981 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:32.013523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4e553-85fd-4c30-8d21-421adcd79981 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:32.014847 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4e553-85fd-4c30-8d21-421adcd79981 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:32.019799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 513/1003] 10.222.0.81 () {56 vars in 1122 bytes} [Mon Sep 19 11:11:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:32.025455 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4e553-85fd-4c30-8d21-421adcd79981 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:32.082779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3fb3b-d2ba-46d4-9146-1be555b2fc35 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:32.118858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9345fa-fca7-4133-bd60-079d5d47713d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:32.119459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9345fa-fca7-4133-bd60-079d5d47713d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:32.119734 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9345fa-fca7-4133-bd60-079d5d47713d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:32.120072 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9345fa-fca7-4133-bd60-079d5d47713d None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:32.120692 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9345fa-fca7-4133-bd60-079d5d47713d None admin] RBAC: Adding query filter params (name=image) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:32.120996 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9345fa-fca7-4133-bd60-079d5d47713d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:32.122367 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9345fa-fca7-4133-bd60-079d5d47713d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:32.127834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 491/1004] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:32.132723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4e553-85fd-4c30-8d21-421adcd79981 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:32.133981 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9345fa-fca7-4133-bd60-079d5d47713d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:32.171290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8fe0f-9ce2-4a7e-b96d-6bc3b18f6433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:32.171915 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8fe0f-9ce2-4a7e-b96d-6bc3b18f6433 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:32.172191 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8fe0f-9ce2-4a7e-b96d-6bc3b18f6433 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:32.172459 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8fe0f-9ce2-4a7e-b96d-6bc3b18f6433 None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:32.173079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8fe0f-9ce2-4a7e-b96d-6bc3b18f6433 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:32.173375 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8fe0f-9ce2-4a7e-b96d-6bc3b18f6433 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:32.175013 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8fe0f-9ce2-4a7e-b96d-6bc3b18f6433 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:32.185433 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 514/1005] 10.222.0.81 () {56 vars in 1122 bytes} [Mon Sep 19 11:11:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:32.187395 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8fe0f-9ce2-4a7e-b96d-6bc3b18f6433 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:32.209692 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:32.215735 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:32.245126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9345fa-fca7-4133-bd60-079d5d47713d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:32.281510 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385052e1-8771-449a-96e7-a89b0068b79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:32.282119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385052e1-8771-449a-96e7-a89b0068b79f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:32.282398 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385052e1-8771-449a-96e7-a89b0068b79f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:32.282667 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385052e1-8771-449a-96e7-a89b0068b79f None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:32.283291 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385052e1-8771-449a-96e7-a89b0068b79f None admin] RBAC: Adding query filter params (type=image) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:32.283589 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385052e1-8771-449a-96e7-a89b0068b79f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:32.285058 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385052e1-8771-449a-96e7-a89b0068b79f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:32.299793 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 492/1006] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:32 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:32.324896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8fe0f-9ce2-4a7e-b96d-6bc3b18f6433 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:32.364067 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afb13e-f34a-4040-bbf0-b1eb8ad52581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:32.364786 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afb13e-f34a-4040-bbf0-b1eb8ad52581 None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:32.365137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afb13e-f34a-4040-bbf0-b1eb8ad52581 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:32.365503 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afb13e-f34a-4040-bbf0-b1eb8ad52581 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:32.366278 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67afb13e-f34a-4040-bbf0-b1eb8ad52581 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:32.367737 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67afb13e-f34a-4040-bbf0-b1eb8ad52581 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:32.387489 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 515/1007] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:11:32 2022] POST /identity/v3/endpoints => generated 325 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:11:33.720931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9dec9f-a924-4cd2-9bda-14f89568075d None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:33.721464 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9dec9f-a924-4cd2-9bda-14f89568075d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:33.721464 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9dec9f-a924-4cd2-9bda-14f89568075d None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:33.722958 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 493/1008] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:33.728494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56b440-63a4-457c-b768-943636ab81f5 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:33.728889 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56b440-63a4-457c-b768-943636ab81f5 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:33.729070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56b440-63a4-457c-b768-943636ab81f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:33.747695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-ec56b440-63a4-457c-b768-943636ab81f5 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:33.748905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ec56b440-63a4-457c-b768-943636ab81f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:33.815728 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:33.828278 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 516/1009] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:33 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:34.088479 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9dec9f-a924-4cd2-9bda-14f89568075d None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:34.190829 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9dec9f-a924-4cd2-9bda-14f89568075d None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:34.225159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab7c51-ba01-4e35-b133-c675faf111b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:34.226318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab7c51-ba01-4e35-b133-c675faf111b1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:34.226603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab7c51-ba01-4e35-b133-c675faf111b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:34.226857 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab7c51-ba01-4e35-b133-c675faf111b1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:34.227432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab7c51-ba01-4e35-b133-c675faf111b1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:34.228825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab7c51-ba01-4e35-b133-c675faf111b1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:34.230589 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:34.239114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-ebab7c51-ba01-4e35-b133-c675faf111b1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:11:34.239114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:34.239114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:34.239114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:34.239114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:34.239114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:34.239114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:34.239114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:34.239114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:11:34.239114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:34.239114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:34.239114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:34.239114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:34.239114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:34.239114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:34.239114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:34.239114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:34.240196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:34.240196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:34.240196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:34.240196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:34.240196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:34.240196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:34.240196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:34.240196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:34.240196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:34.240196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:34.240196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:34.240196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:34.240196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:34.240196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:34.240196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:34.240196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:34.241088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:34.241088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:34.241088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:34.241088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:34.241088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:34.241088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:34.241088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:34.241088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:34.241088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:34.241088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:34.241088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:34.241088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:34.241088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 11:11:34.241088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:34.243919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 494/1010] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:11:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:34.246861 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56b440-63a4-457c-b768-943636ab81f5 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:34.345666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56b440-63a4-457c-b768-943636ab81f5 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:34.382535 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5ef2c-9c84-4d2b-950b-34c3192ef1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:34.383164 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5ef2c-9c84-4d2b-950b-34c3192ef1c7 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:34.383442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5ef2c-9c84-4d2b-950b-34c3192ef1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:34.383713 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5ef2c-9c84-4d2b-950b-34c3192ef1c7 None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:34.384626 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee5ef2c-9c84-4d2b-950b-34c3192ef1c7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:34.384940 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee5ef2c-9c84-4d2b-950b-34c3192ef1c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:34.386363 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee5ef2c-9c84-4d2b-950b-34c3192ef1c7 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:34.396924 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 517/1011] 10.222.0.81 () {56 vars in 1112 bytes} [Mon Sep 19 11:11:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:34.398759 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab7c51-ba01-4e35-b133-c675faf111b1 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:34.498116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f6e8f-844a-4751-b81a-2b3493266c9d None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:34.498460 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f6e8f-844a-4751-b81a-2b3493266c9d None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:34.498683 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f6e8f-844a-4751-b81a-2b3493266c9d None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:34.500034 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 518/1012] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:34.504858 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb633722-2379-4cc1-8ae0-17956d9b61c8 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:34.505300 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb633722-2379-4cc1-8ae0-17956d9b61c8 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:34.505574 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb633722-2379-4cc1-8ae0-17956d9b61c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:34.508315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab7c51-ba01-4e35-b133-c675faf111b1 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:34.528138 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-bb633722-2379-4cc1-8ae0-17956d9b61c8 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:34.529330 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-bb633722-2379-4cc1-8ae0-17956d9b61c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:34.546595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c5209-6d3c-4fc7-93dc-419c954020a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:34.547493 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c5209-6d3c-4fc7-93dc-419c954020a5 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:34.547790 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c5209-6d3c-4fc7-93dc-419c954020a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:34.548077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c5209-6d3c-4fc7-93dc-419c954020a5 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:34.548707 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c5209-6d3c-4fc7-93dc-419c954020a5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:34.549006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c5209-6d3c-4fc7-93dc-419c954020a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:34.550852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c5209-6d3c-4fc7-93dc-419c954020a5 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:34.556587 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 495/1013] 10.222.0.81 () {56 vars in 1112 bytes} [Mon Sep 19 11:11:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:34.561912 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c5209-6d3c-4fc7-93dc-419c954020a5 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:34.591041 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:34.600113 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 519/1014] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:34 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:11:34.602010 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886647ff-6893-4184-aae6-277db295bd27 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:34.602290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886647ff-6893-4184-aae6-277db295bd27 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:34.602563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886647ff-6893-4184-aae6-277db295bd27 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:34.603799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 520/1015] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:34.609013 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22ce65-07fe-4cdd-9927-539759132cc1 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:34.609294 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22ce65-07fe-4cdd-9927-539759132cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:34.609592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22ce65-07fe-4cdd-9927-539759132cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:34.627118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-ca22ce65-07fe-4cdd-9927-539759132cc1 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:34.628346 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ca22ce65-07fe-4cdd-9927-539759132cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:34.630631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:34.681286 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c5209-6d3c-4fc7-93dc-419c954020a5 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:34.703779 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 521/1016] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:34 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:11:34.725915 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad611d1-a123-4f70-84f9-5b476c0ec461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:34.726596 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad611d1-a123-4f70-84f9-5b476c0ec461 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:34.726964 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad611d1-a123-4f70-84f9-5b476c0ec461 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:34.727303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad611d1-a123-4f70-84f9-5b476c0ec461 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:34.728091 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad611d1-a123-4f70-84f9-5b476c0ec461 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:34.729612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad611d1-a123-4f70-84f9-5b476c0ec461 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:34.756213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 496/1017] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:11:34 2022] POST /identity/v3/endpoints => generated 334 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:11:34.874622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22ce65-07fe-4cdd-9927-539759132cc1 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:34.977744 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad611d1-a123-4f70-84f9-5b476c0ec461 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:34.981791 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22ce65-07fe-4cdd-9927-539759132cc1 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:35.008368 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:35.031437 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3664e7-8501-49a1-8dd2-cdf3d058eaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:35.032099 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3664e7-8501-49a1-8dd2-cdf3d058eaf2 None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:35.032375 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3664e7-8501-49a1-8dd2-cdf3d058eaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:35.032691 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3664e7-8501-49a1-8dd2-cdf3d058eaf2 None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:35.033296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3664e7-8501-49a1-8dd2-cdf3d058eaf2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:35.034657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3664e7-8501-49a1-8dd2-cdf3d058eaf2 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:35.051744 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 522/1018] 10.222.0.81 () {60 vars in 1141 bytes} [Mon Sep 19 11:11:34 2022] POST /identity/v3/services => generated 242 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:11:35.094897 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad611d1-a123-4f70-84f9-5b476c0ec461 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:35.114651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:35.144019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44cf725-4b10-4757-8bc8-c61b1b2a022d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:35.144641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cf725-4b10-4757-8bc8-c61b1b2a022d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:35.144922 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cf725-4b10-4757-8bc8-c61b1b2a022d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:35.145192 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cf725-4b10-4757-8bc8-c61b1b2a022d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:35.146352 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44cf725-4b10-4757-8bc8-c61b1b2a022d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:35.148435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44cf725-4b10-4757-8bc8-c61b1b2a022d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:35.151794 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 497/1019] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:35.156081 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3664e7-8501-49a1-8dd2-cdf3d058eaf2 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:35.264739 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3664e7-8501-49a1-8dd2-cdf3d058eaf2 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:35.298579 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa2d63-f494-4e11-8903-9799f4696111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:35.299182 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa2d63-f494-4e11-8903-9799f4696111 None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:35.299459 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa2d63-f494-4e11-8903-9799f4696111 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:35.299729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa2d63-f494-4e11-8903-9799f4696111 None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:35.300422 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa2d63-f494-4e11-8903-9799f4696111 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:35.301843 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa2d63-f494-4e11-8903-9799f4696111 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:35.330742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 523/1020] 10.222.0.81 () {60 vars in 1141 bytes} [Mon Sep 19 11:11:35 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:11:37.028528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16797c6-63fa-4337-a094-1b2b9697e3d6 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:37.029032 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16797c6-63fa-4337-a094-1b2b9697e3d6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:37.029032 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16797c6-63fa-4337-a094-1b2b9697e3d6 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:37.030348 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 498/1021] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:37.035392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011f4a6-6f0e-43e8-a887-47764188267a None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:37.035929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011f4a6-6f0e-43e8-a887-47764188267a None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:37.036020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011f4a6-6f0e-43e8-a887-47764188267a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:37.054602 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-b011f4a6-6f0e-43e8-a887-47764188267a None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:37.055819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b011f4a6-6f0e-43e8-a887-47764188267a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:37.082370 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:37.127805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 524/1022] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:37.259865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51062d-5304-45c4-b856-f4f0becf8a18 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:37.260184 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51062d-5304-45c4-b856-f4f0becf8a18 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:37.260432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51062d-5304-45c4-b856-f4f0becf8a18 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:37.261911 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 499/1023] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:37.267075 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6b09c-bf74-4d2c-a6af-cd54dffcf6b5 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:37.267369 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6b09c-bf74-4d2c-a6af-cd54dffcf6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:37.267646 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6b09c-bf74-4d2c-a6af-cd54dffcf6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:37.287405 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-6bf6b09c-bf74-4d2c-a6af-cd54dffcf6b5 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:37.289418 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-6bf6b09c-bf74-4d2c-a6af-cd54dffcf6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:37.357088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 525/1024] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:37.392871 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51062d-5304-45c4-b856-f4f0becf8a18 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:37.501607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51062d-5304-45c4-b856-f4f0becf8a18 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:37.533289 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae2217-1677-4d7e-8323-48485fbc6eb1 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:37.533641 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae2217-1677-4d7e-8323-48485fbc6eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:37.533853 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae2217-1677-4d7e-8323-48485fbc6eb1 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:37.535174 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 526/1025] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:37.536732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562c70e-0802-4661-af53-601f5cda0765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:37.537644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562c70e-0802-4661-af53-601f5cda0765 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:37.538151 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562c70e-0802-4661-af53-601f5cda0765 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:37.538609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562c70e-0802-4661-af53-601f5cda0765 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:37.540100 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd11898-2476-4077-a947-e482b7ea034c None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:37.540385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd11898-2476-4077-a947-e482b7ea034c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:37.540654 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd11898-2476-4077-a947-e482b7ea034c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:37.549938 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562c70e-0802-4661-af53-601f5cda0765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:11:37.555662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-5562c70e-0802-4661-af53-601f5cda0765 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:11:37.555662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:37.555662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:37.555662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:37.555662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:37.555662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:37.555662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:37.555662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:37.555662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:37.555662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:37.555662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:37.555662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:37.555662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:11:37.555662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:11:37.555662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:11:37.555662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:11:37.556795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:11:37.556795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:11:37.556795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:11:37.556795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:11:37.556795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:37.556795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:37.556795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:37.556795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:37.556795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:37.556795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:37.556795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:37.556795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:37.556795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:37.556795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:37.556795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:37.556795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:37.557879 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:37.557879 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:37.557879 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:37.557879 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:37.557879 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:11:37.557879 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:11:37.557879 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:11:37.557879 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:11:37.557879 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:11:37.557879 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:11:37.557879 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:11:37.557879 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:37.558749 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-7fd11898-2476-4077-a947-e482b7ea034c None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:37.559964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-7fd11898-2476-4077-a947-e482b7ea034c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:37.560121 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 500/1026] 10.222.0.81 () {56 vars in 1120 bytes} [Mon Sep 19 11:11:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:37.565625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562c70e-0802-4661-af53-601f5cda0765 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:37.613230 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:37.634944 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 527/1027] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:11:37.635900 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd11898-2476-4077-a947-e482b7ea034c None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:37.662897 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:37.681900 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562c70e-0802-4661-af53-601f5cda0765 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:37.717421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef0280-53a7-4860-bd72-db134a144273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:37.718014 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef0280-53a7-4860-bd72-db134a144273 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:37.718294 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef0280-53a7-4860-bd72-db134a144273 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:37.718566 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef0280-53a7-4860-bd72-db134a144273 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:37.719224 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef0280-53a7-4860-bd72-db134a144273 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:37.719531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef0280-53a7-4860-bd72-db134a144273 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:37.720937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef0280-53a7-4860-bd72-db134a144273 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:37.728891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 501/1028] 10.222.0.81 () {56 vars in 1115 bytes} [Mon Sep 19 11:11:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:37.736892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef0280-53a7-4860-bd72-db134a144273 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:37.744413 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd11898-2476-4077-a947-e482b7ea034c None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:37.780970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b23c9f-45d0-4f03-8487-b58f00e2d0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:37.781600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b23c9f-45d0-4f03-8487-b58f00e2d0fa None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:37.781877 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b23c9f-45d0-4f03-8487-b58f00e2d0fa None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:37.782154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b23c9f-45d0-4f03-8487-b58f00e2d0fa None admin] PATH_INFO: `/v3/services/image` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:37.782726 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b23c9f-45d0-4f03-8487-b58f00e2d0fa None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:37.784246 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b23c9f-45d0-4f03-8487-b58f00e2d0fa None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:37.794411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-a8b23c9f-45d0-4f03-8487-b58f00e2d0fa None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:11:37.794411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:37.794411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:37.794411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:37.794411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:37.794411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:37.794411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:37.794411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:37.794411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:11:37.794411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:37.794411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:37.794411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:37.794411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:37.794411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:37.794411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:37.794411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:37.794411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:37.795380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:37.795380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:37.795380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:37.795380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:37.795380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:37.795380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:37.795380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:37.795380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:37.795380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:37.795380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:37.795380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:37.795380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:37.795380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:37.795380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:37.795380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:37.795380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:37.800796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:37.800796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:37.800796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:37.800796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:37.800796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:37.800796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:37.800796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:37.800796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:37.800796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:37.800796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:37.800796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:37.800796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:37.800796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:11:37.800796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:37.800796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 528/1029] 10.222.0.81 () {56 vars in 1105 bytes} [Mon Sep 19 11:11:37 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:37.801821 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b23c9f-45d0-4f03-8487-b58f00e2d0fa None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:37.845496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef0280-53a7-4860-bd72-db134a144273 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:37.848477 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:37.885520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befcc59-df8a-4bcc-bb99-0d83823427e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:37.886660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befcc59-df8a-4bcc-bb99-0d83823427e1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:37.886940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befcc59-df8a-4bcc-bb99-0d83823427e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:37.887469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befcc59-df8a-4bcc-bb99-0d83823427e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:37.889828 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:37.903954 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5befcc59-df8a-4bcc-bb99-0d83823427e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:37.905402 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5befcc59-df8a-4bcc-bb99-0d83823427e1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:37.906076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b23c9f-45d0-4f03-8487-b58f00e2d0fa None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:37.919853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-5befcc59-df8a-4bcc-bb99-0d83823427e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:37.919853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:37.919853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:37.919853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:37.919853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:37.919853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:37.919853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:37.919853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:37.919853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:37.919853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:37.919853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:37.919853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:37.919853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:37.919853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:37.919853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:37.919853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:37.921101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:37.921101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:37.921101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:37.921101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:37.921101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:37.921101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:37.921101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:37.921101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:37.921101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:37.921101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:37.921101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:37.921101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:37.921101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:37.921101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:37.921101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:37.921101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:37.921101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:37.922363 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:37.922363 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:37.922363 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:37.922363 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 502/1030] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:37.922363 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befcc59-df8a-4bcc-bb99-0d83823427e1 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:37.942835 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed5156-2a9f-4154-9ad4-b58a330feda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:37.943434 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed5156-2a9f-4154-9ad4-b58a330feda6 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:37.943780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed5156-2a9f-4154-9ad4-b58a330feda6 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:37.944719 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed5156-2a9f-4154-9ad4-b58a330feda6 None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:37.945350 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed5156-2a9f-4154-9ad4-b58a330feda6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:37.945656 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed5156-2a9f-4154-9ad4-b58a330feda6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:37.947006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed5156-2a9f-4154-9ad4-b58a330feda6 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:37.952091 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 529/1031] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:37.953165 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed5156-2a9f-4154-9ad4-b58a330feda6 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:38.023843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befcc59-df8a-4bcc-bb99-0d83823427e1 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:38.050574 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed5156-2a9f-4154-9ad4-b58a330feda6 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:38.056194 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9dfa3-5fb8-4958-9af2-bd1606a39870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:38.056808 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9dfa3-5fb8-4958-9af2-bd1606a39870 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:38.057086 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9dfa3-5fb8-4958-9af2-bd1606a39870 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:38.057355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9dfa3-5fb8-4958-9af2-bd1606a39870 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:38.058451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c9dfa3-5fb8-4958-9af2-bd1606a39870 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:38.059915 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c9dfa3-5fb8-4958-9af2-bd1606a39870 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:38.061472 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 503/1032] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:38.062218 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9dfa3-5fb8-4958-9af2-bd1606a39870 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:38.085264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7d774-a4be-4e11-b389-e6d04211742d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:38.085866 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7d774-a4be-4e11-b389-e6d04211742d None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:38.086141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7d774-a4be-4e11-b389-e6d04211742d None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:38.086405 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7d774-a4be-4e11-b389-e6d04211742d None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:38.087019 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7d774-a4be-4e11-b389-e6d04211742d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:38.087319 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7d774-a4be-4e11-b389-e6d04211742d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:38.088696 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7d774-a4be-4e11-b389-e6d04211742d None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:38.090629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:38.100593 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 530/1033] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:38.101374 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7d774-a4be-4e11-b389-e6d04211742d None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:38.114997 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:38.163701 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9dfa3-5fb8-4958-9af2-bd1606a39870 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:38.196798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a17f8-0563-49ef-827b-944e1afded8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:38.197387 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a17f8-0563-49ef-827b-944e1afded8c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:38.197664 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a17f8-0563-49ef-827b-944e1afded8c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:38.197931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a17f8-0563-49ef-827b-944e1afded8c None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:38.198550 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a17f8-0563-49ef-827b-944e1afded8c None admin] RBAC: Adding query filter params (type=image) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:38.198848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a17f8-0563-49ef-827b-944e1afded8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:38.200239 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a17f8-0563-49ef-827b-944e1afded8c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:38.202317 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7d774-a4be-4e11-b389-e6d04211742d None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:38.205101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 504/1034] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:38.205869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a17f8-0563-49ef-827b-944e1afded8c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:38.236865 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c32a2-0dee-47eb-a90d-83f275d84487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:38.237457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c32a2-0dee-47eb-a90d-83f275d84487 None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:38.237732 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c32a2-0dee-47eb-a90d-83f275d84487 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:38.237999 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c32a2-0dee-47eb-a90d-83f275d84487 None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:38.238640 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c32a2-0dee-47eb-a90d-83f275d84487 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:38.240064 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c32a2-0dee-47eb-a90d-83f275d84487 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:38.262431 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 531/1035] 10.222.0.81 () {60 vars in 1132 bytes} [Mon Sep 19 11:11:38 2022] POST /identity/v3/users => generated 285 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:11:38.263465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c32a2-0dee-47eb-a90d-83f275d84487 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:38.313261 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:38.327071 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a17f8-0563-49ef-827b-944e1afded8c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:38.339130 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:38.366429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f4daf-cea7-498f-b939-4fa1ac0043b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:38.367071 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f4daf-cea7-498f-b939-4fa1ac0043b9 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:38.367345 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f4daf-cea7-498f-b939-4fa1ac0043b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:38.367610 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f4daf-cea7-498f-b939-4fa1ac0043b9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:38.375884 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f4daf-cea7-498f-b939-4fa1ac0043b9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:38.377336 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f4daf-cea7-498f-b939-4fa1ac0043b9 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:38.382131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c32a2-0dee-47eb-a90d-83f275d84487 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:38.389317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-718f4daf-cea7-498f-b939-4fa1ac0043b9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:11:38.389317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:38.389317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:38.389317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:38.389317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:38.389317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:38.389317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:38.389317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:38.389317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:38.389317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:38.389317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:38.389317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:38.389317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:11:38.389317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:11:38.389317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:11:38.389317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:11:38.390397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:38.390397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:38.390397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:11:38.390397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:38.390397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:11:38.390397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:38.390397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:38.390397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:38.390397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:38.390397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:38.390397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:38.390397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:38.390397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:38.390397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:38.390397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:38.390397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:38.390397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:38.390397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:38.391370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:38.391370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:38.391370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:11:38.391370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:11:38.391370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:11:38.391370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:11:38.391370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:11:38.391370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:11:38.391370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:11:38.391370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:38.395494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 505/1036] 10.222.0.81 () {56 vars in 1093 bytes} [Mon Sep 19 11:11:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:38.396479 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f4daf-cea7-498f-b939-4fa1ac0043b9 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:38.426748 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fdf29-fe71-4973-9007-c6f3c64b1af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:38.427368 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fdf29-fe71-4973-9007-c6f3c64b1af4 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:38.427645 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fdf29-fe71-4973-9007-c6f3c64b1af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:38.427946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fdf29-fe71-4973-9007-c6f3c64b1af4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:38.428684 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135fdf29-fe71-4973-9007-c6f3c64b1af4 None admin] RBAC: Adding query filter params (interface=public, service_id=2e0cfce2dede4d9e8183a87aa9bd212b, region_id=RegionOne) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:38.428990 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135fdf29-fe71-4973-9007-c6f3c64b1af4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:38.430355 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135fdf29-fe71-4973-9007-c6f3c64b1af4 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:38.435581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 532/1037] 10.222.0.81 () {56 vars in 1251 bytes} [Mon Sep 19 11:11:38 2022] GET /identity/v3/endpoints?service_id=2e0cfce2dede4d9e8183a87aa9bd212b&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:38.441391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fdf29-fe71-4973-9007-c6f3c64b1af4 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:38.517384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f4daf-cea7-498f-b939-4fa1ac0043b9 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:38.556075 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67f51d-a534-4aa6-b1ea-3522c2e8b38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:38.556707 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67f51d-a534-4aa6-b1ea-3522c2e8b38d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:38.556989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67f51d-a534-4aa6-b1ea-3522c2e8b38d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:38.557253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67f51d-a534-4aa6-b1ea-3522c2e8b38d None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:38.557990 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f67f51d-a534-4aa6-b1ea-3522c2e8b38d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:38.558297 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f67f51d-a534-4aa6-b1ea-3522c2e8b38d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:38.559571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fdf29-fe71-4973-9007-c6f3c64b1af4 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:38.559813 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f67f51d-a534-4aa6-b1ea-3522c2e8b38d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:38.583440 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 506/1038] 10.222.0.81 () {56 vars in 1133 bytes} [Mon Sep 19 11:11:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:38.585187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67f51d-a534-4aa6-b1ea-3522c2e8b38d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:38.591006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:38.595744 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:38.597893 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d2041-1a3e-452b-99d6-5b004642ac08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:38.598560 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d2041-1a3e-452b-99d6-5b004642ac08 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:38.598879 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d2041-1a3e-452b-99d6-5b004642ac08 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:38.599177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d2041-1a3e-452b-99d6-5b004642ac08 None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:38.599837 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d2041-1a3e-452b-99d6-5b004642ac08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:38.601309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d2041-1a3e-452b-99d6-5b004642ac08 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:38.616299 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 533/1039] 10.222.0.81 () {56 vars in 1087 bytes} [Mon Sep 19 11:11:38 2022] GET /identity/v3/services => generated 1957 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:11:38.703683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67f51d-a534-4aa6-b1ea-3522c2e8b38d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:38.740515 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d90971-02cc-422b-8217-a8766f9f384b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:38.741703 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d90971-02cc-422b-8217-a8766f9f384b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:38.741984 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d90971-02cc-422b-8217-a8766f9f384b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:38.742257 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d90971-02cc-422b-8217-a8766f9f384b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:38.821671 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d90971-02cc-422b-8217-a8766f9f384b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:38.823137 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d90971-02cc-422b-8217-a8766f9f384b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:38.831933 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:38.927028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 507/1040] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:11:38 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 11:11:38.933241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d2041-1a3e-452b-99d6-5b004642ac08 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:39.034639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d2041-1a3e-452b-99d6-5b004642ac08 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:39.068854 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc25897-86db-4567-84eb-a3c26098fc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:39.069626 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc25897-86db-4567-84eb-a3c26098fc88 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:39.069838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc25897-86db-4567-84eb-a3c26098fc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:39.070074 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc25897-86db-4567-84eb-a3c26098fc88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:39.081209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc25897-86db-4567-84eb-a3c26098fc88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:39.082646 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc25897-86db-4567-84eb-a3c26098fc88 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:39.098523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-dfc25897-86db-4567-84eb-a3c26098fc88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:39.098523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:39.098523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:39.098523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:39.098523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:39.098523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:39.098523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:39.098523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:39.098523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:39.098523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:39.098523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:39.098523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:39.098523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:39.098523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:39.098523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:39.098523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:39.099509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:39.099509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:39.099509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:39.099509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:39.099509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:39.099509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:39.099509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:39.099509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:39.099509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:39.099509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:39.099509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:39.099509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:39.099509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:39.099509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:39.099509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:39.099509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:39.099509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:39.101432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:39.101432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:39.101432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:39.104073 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 534/1041] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:39.106112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d90971-02cc-422b-8217-a8766f9f384b None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:39.205493 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d90971-02cc-422b-8217-a8766f9f384b None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:39.214813 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:39.243679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29131f-de4d-4c43-8617-7084684cd0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:39.244599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29131f-de4d-4c43-8617-7084684cd0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:39.245092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29131f-de4d-4c43-8617-7084684cd0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:39.245563 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29131f-de4d-4c43-8617-7084684cd0c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:39.246410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29131f-de4d-4c43-8617-7084684cd0c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:39.246912 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29131f-de4d-4c43-8617-7084684cd0c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:39.248481 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29131f-de4d-4c43-8617-7084684cd0c7 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:39.264406 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 508/1042] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:39.266243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc25897-86db-4567-84eb-a3c26098fc88 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:39.315997 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:39.371009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc25897-86db-4567-84eb-a3c26098fc88 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:39.404569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c25d4b-a27d-4498-9759-369fa595d3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:39.405194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c25d4b-a27d-4498-9759-369fa595d3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:39.405475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c25d4b-a27d-4498-9759-369fa595d3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:39.405748 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c25d4b-a27d-4498-9759-369fa595d3d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:39.412337 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c25d4b-a27d-4498-9759-369fa595d3d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:39.413903 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c25d4b-a27d-4498-9759-369fa595d3d2 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:39.428603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-25c25d4b-a27d-4498-9759-369fa595d3d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:39.428603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:39.428603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:39.428603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:39.428603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:39.428603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:39.428603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:39.428603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:39.428603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:39.428603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:39.428603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:39.428603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:39.428603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:11:39.428603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:11:39.428603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:11:39.429639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:11:39.429639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:39.429639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:39.429639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:39.429639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:39.429639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:39.429639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:39.429639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:39.429639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:39.429639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:39.429639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:39.429639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:39.429639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:39.429639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:39.429639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:39.429639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:39.429639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:39.430548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:11:39.430548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:11:39.430548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:11:39.430548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:11:39.430548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:11:39.430548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:11:39.430548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:39.430548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:39.432091 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 535/1043] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:11:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:39.435680 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29131f-de4d-4c43-8617-7084684cd0c7 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:39.541397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29131f-de4d-4c43-8617-7084684cd0c7 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:39.574779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90208375-4525-4e1d-98b2-135f866feb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:39.575382 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90208375-4525-4e1d-98b2-135f866feb53 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:39.575660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90208375-4525-4e1d-98b2-135f866feb53 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:39.575976 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90208375-4525-4e1d-98b2-135f866feb53 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:39.576666 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90208375-4525-4e1d-98b2-135f866feb53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:39.576971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90208375-4525-4e1d-98b2-135f866feb53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:39.578646 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90208375-4525-4e1d-98b2-135f866feb53 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:39.591232 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 509/1044] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:11:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:39.597064 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c25d4b-a27d-4498-9759-369fa595d3d2 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:39.706153 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c25d4b-a27d-4498-9759-369fa595d3d2 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:39.713545 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:39.740647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1adea88-e236-4e25-bf40-9b820feb8017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:39.741270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adea88-e236-4e25-bf40-9b820feb8017 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:39.741545 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adea88-e236-4e25-bf40-9b820feb8017 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:39.741810 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1adea88-e236-4e25-bf40-9b820feb8017 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:39.742522 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1adea88-e236-4e25-bf40-9b820feb8017 None admin] RBAC: Adding query filter params (role.id=7689e410cf25472397444b35809be2c7, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=5abd70a18ccd407bafe3b61c8870986e) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:39.742828 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1adea88-e236-4e25-bf40-9b820feb8017 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:39.744265 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1adea88-e236-4e25-bf40-9b820feb8017 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:39.755846 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 536/1045] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:11:39 2022] GET /identity/v3/role_assignments?user.id=5abd70a18ccd407bafe3b61c8870986e&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=7689e410cf25472397444b35809be2c7 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:40.500629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6de27-84e0-48d3-8af2-f06c1e5188d5 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:40.501192 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6de27-84e0-48d3-8af2-f06c1e5188d5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:40.501192 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6de27-84e0-48d3-8af2-f06c1e5188d5 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:40.502573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 510/1046] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:40.508308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea3903-2fa9-4f19-a363-818e6524b219 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:40.508733 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea3903-2fa9-4f19-a363-818e6524b219 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:40.508899 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea3903-2fa9-4f19-a363-818e6524b219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:40.526836 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-5aea3903-2fa9-4f19-a363-818e6524b219 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:40.528055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5aea3903-2fa9-4f19-a363-818e6524b219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:40.597655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 537/1047] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:40.761629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a54cd-a828-481b-8793-4c0b29ddb2e6 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:40.761925 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a54cd-a828-481b-8793-4c0b29ddb2e6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:40.762103 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a54cd-a828-481b-8793-4c0b29ddb2e6 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:40.763488 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 511/1048] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:40.768548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c0991-774d-4fcc-8cd9-5caee58f5818 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:40.768836 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c0991-774d-4fcc-8cd9-5caee58f5818 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:40.769108 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c0991-774d-4fcc-8cd9-5caee58f5818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:40.788394 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-560c0991-774d-4fcc-8cd9-5caee58f5818 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:40.789584 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-560c0991-774d-4fcc-8cd9-5caee58f5818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:40.859860 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 538/1049] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:40.874471 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a54cd-a828-481b-8793-4c0b29ddb2e6 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:40.901982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:40.978447 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a54cd-a828-481b-8793-4c0b29ddb2e6 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:41.012589 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617e48e-5654-45f7-83c6-7fc4611158bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:41.013143 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617e48e-5654-45f7-83c6-7fc4611158bc None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:41.013423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617e48e-5654-45f7-83c6-7fc4611158bc None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:41.013696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617e48e-5654-45f7-83c6-7fc4611158bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:41.024007 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617e48e-5654-45f7-83c6-7fc4611158bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:11:41.028865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-6617e48e-5654-45f7-83c6-7fc4611158bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:11:41.028865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:41.028865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:41.028865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:41.028865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:41.028865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:41.028865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:41.028865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:41.028865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:41.028865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:41.028865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:41.028865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:41.028865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:11:41.028865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:11:41.028865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:11:41.028865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:11:41.029779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:11:41.029779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:11:41.029779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:11:41.029779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:11:41.029779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:41.029779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:41.029779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:41.029779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:41.029779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:41.029779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:41.029779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:41.029779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:41.029779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:41.029779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:41.029779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:41.029779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:41.030631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:41.030631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:41.030631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:41.030631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:41.030631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:11:41.030631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:11:41.030631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:11:41.030631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:11:41.030631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:11:41.030631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:11:41.030631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:11:41.030631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:41.035078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 512/1050] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:11:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:41.035981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c0991-774d-4fcc-8cd9-5caee58f5818 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:41.066324 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:41.125064 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617e48e-5654-45f7-83c6-7fc4611158bc None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:41.138506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c0991-774d-4fcc-8cd9-5caee58f5818 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:41.173774 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3bdacf-4b38-4d1a-871d-eac9e3005a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:41.174373 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bdacf-4b38-4d1a-871d-eac9e3005a42 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:41.174655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bdacf-4b38-4d1a-871d-eac9e3005a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:41.174925 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bdacf-4b38-4d1a-871d-eac9e3005a42 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:41.175578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3bdacf-4b38-4d1a-871d-eac9e3005a42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:41.175909 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3bdacf-4b38-4d1a-871d-eac9e3005a42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:41.177294 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3bdacf-4b38-4d1a-871d-eac9e3005a42 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:41.185229 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 539/1051] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:11:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:41.191475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3bdacf-4b38-4d1a-871d-eac9e3005a42 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:41.223966 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:41.241836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617e48e-5654-45f7-83c6-7fc4611158bc None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:41.277335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa55bc-8c4b-49e5-a050-359f5f3f4528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:41.277894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa55bc-8c4b-49e5-a050-359f5f3f4528 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:41.278158 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa55bc-8c4b-49e5-a050-359f5f3f4528 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:41.278428 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa55bc-8c4b-49e5-a050-359f5f3f4528 None admin] PATH_INFO: `/v3/services/image` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:41.279006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa55bc-8c4b-49e5-a050-359f5f3f4528 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:41.280483 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa55bc-8c4b-49e5-a050-359f5f3f4528 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:41.289308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-e3aa55bc-8c4b-49e5-a050-359f5f3f4528 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:11:41.289308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:41.289308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:11:41.289308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:11:41.289308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:11:41.289308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:11:41.289308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:11:41.289308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:41.289308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:11:41.289308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:41.289308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:11:41.289308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:41.289308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:41.289308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:41.289308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:41.289308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:41.289308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:41.290496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:41.290496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:41.290496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:41.290496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:41.290496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:41.290496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:41.290496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:11:41.290496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:11:41.290496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:11:41.290496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:11:41.290496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:41.290496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:41.290496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:41.290496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:41.290496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:41.290496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:41.291346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:41.291346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:41.291346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:41.291346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:41.291346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:41.291346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:41.291346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:41.291346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:41.291346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:41.291346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:41.291346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:11:41.291346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:11:41.291346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 11:11:41.291346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:41.291346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 513/1052] 10.222.0.81 () {56 vars in 1105 bytes} [Mon Sep 19 11:11:41 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:41.296212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:41.296348 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa55bc-8c4b-49e5-a050-359f5f3f4528 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:41.307536 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3bdacf-4b38-4d1a-871d-eac9e3005a42 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:41.346249 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7ca87-8d69-40f0-a368-9de321b3039f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:41.347030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7ca87-8d69-40f0-a368-9de321b3039f None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:41.347271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7ca87-8d69-40f0-a368-9de321b3039f None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:41.347539 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7ca87-8d69-40f0-a368-9de321b3039f None admin] PATH_INFO: `/v3/users/d4c2e44d11a74ced925a3d7e24aa92e5` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:41.361934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7ca87-8d69-40f0-a368-9de321b3039f None admin] RBAC: Authorizing `identity:get_user(user_id=d4c2e44d11a74ced925a3d7e24aa92e5)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:41.363357 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7ca87-8d69-40f0-a368-9de321b3039f None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:41.365167 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 540/1053] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:11:41 2022] GET /identity/v3/users/d4c2e44d11a74ced925a3d7e24aa92e5 => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:41.371311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7ca87-8d69-40f0-a368-9de321b3039f None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:41.401752 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa55bc-8c4b-49e5-a050-359f5f3f4528 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:41.437875 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550fcac-8a9b-401a-ab6b-df2d0c032652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:41.438481 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550fcac-8a9b-401a-ab6b-df2d0c032652 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:41.438761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550fcac-8a9b-401a-ab6b-df2d0c032652 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:41.439038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550fcac-8a9b-401a-ab6b-df2d0c032652 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:41.439667 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550fcac-8a9b-401a-ab6b-df2d0c032652 None admin] RBAC: Adding query filter params (name=image) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:41.440074 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550fcac-8a9b-401a-ab6b-df2d0c032652 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:41.441449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550fcac-8a9b-401a-ab6b-df2d0c032652 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:41.446635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 514/1054] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:41.452212 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550fcac-8a9b-401a-ab6b-df2d0c032652 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:41.469304 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:41.477855 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7ca87-8d69-40f0-a368-9de321b3039f None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:41.514070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5066f3-a4a2-4ad1-9b8a-c7a385d907db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:41.514721 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5066f3-a4a2-4ad1-9b8a-c7a385d907db None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:41.515065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5066f3-a4a2-4ad1-9b8a-c7a385d907db None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:41.515235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5066f3-a4a2-4ad1-9b8a-c7a385d907db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:41.536325 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:41.561722 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550fcac-8a9b-401a-ab6b-df2d0c032652 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:41.589489 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5066f3-a4a2-4ad1-9b8a-c7a385d907db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:41.590985 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5066f3-a4a2-4ad1-9b8a-c7a385d907db None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:41.597671 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae778b6d-97bd-4630-a932-bf5a78f49a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:41.598281 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae778b6d-97bd-4630-a932-bf5a78f49a67 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:41.598568 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae778b6d-97bd-4630-a932-bf5a78f49a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:41.598840 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae778b6d-97bd-4630-a932-bf5a78f49a67 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:41.599468 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae778b6d-97bd-4630-a932-bf5a78f49a67 None admin] RBAC: Adding query filter params (type=image) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:41.599800 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae778b6d-97bd-4630-a932-bf5a78f49a67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:41.601290 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae778b6d-97bd-4630-a932-bf5a78f49a67 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:41.606654 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 515/1055] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:41.612669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae778b6d-97bd-4630-a932-bf5a78f49a67 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:41.696815 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:41.708652 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 541/1056] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:11:41 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 11:11:41.714905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5066f3-a4a2-4ad1-9b8a-c7a385d907db None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:41.726554 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae778b6d-97bd-4630-a932-bf5a78f49a67 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:41.762633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ed8df-831a-4e86-b36f-a4ec971e93ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:41.763255 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ed8df-831a-4e86-b36f-a4ec971e93ae None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:41.763550 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ed8df-831a-4e86-b36f-a4ec971e93ae None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:41.763862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ed8df-831a-4e86-b36f-a4ec971e93ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:41.764511 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6ed8df-831a-4e86-b36f-a4ec971e93ae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:41.765932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6ed8df-831a-4e86-b36f-a4ec971e93ae None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:41.769539 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:41.788944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 516/1057] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:11:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:11:41.840937 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5066f3-a4a2-4ad1-9b8a-c7a385d907db None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:41.885841 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08b37f-47f6-4d9b-b789-19dbfd82765b None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:41.886131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08b37f-47f6-4d9b-b789-19dbfd82765b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:41.888720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08b37f-47f6-4d9b-b789-19dbfd82765b None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:41.888844 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a4b94-fb0b-475d-a1ab-ab9a8ee2a4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:41.888844 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a4b94-fb0b-475d-a1ab-ab9a8ee2a4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:41.888844 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a4b94-fb0b-475d-a1ab-ab9a8ee2a4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:41.889058 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 517/1058] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:11:41.889148 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a4b94-fb0b-475d-a1ab-ab9a8ee2a4c7 None admin] PATH_INFO: `/v3/projects/c0e051a87db447378722164df3f66964` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:41.889986 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a4b94-fb0b-475d-a1ab-ab9a8ee2a4c7 None admin] RBAC: Authorizing `identity:get_project(project_id=c0e051a87db447378722164df3f66964)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:41.891635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a4b94-fb0b-475d-a1ab-ab9a8ee2a4c7 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:41.893702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 542/1059] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:11:41 2022] GET /identity/v3/projects/c0e051a87db447378722164df3f66964 => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:41.895847 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c539546-319f-4c22-93ec-f531fcc032af None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:41.896266 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c539546-319f-4c22-93ec-f531fcc032af None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:41.896617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c539546-319f-4c22-93ec-f531fcc032af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:41.900107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a4b94-fb0b-475d-a1ab-ab9a8ee2a4c7 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:41.918408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-3c539546-319f-4c22-93ec-f531fcc032af None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:41.919691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3c539546-319f-4c22-93ec-f531fcc032af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:41.998329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 518/1060] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:11:42.013490 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a4b94-fb0b-475d-a1ab-ab9a8ee2a4c7 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:42.028627 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:42.049640 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f263dbe-fba5-49f2-8d21-e9d657142bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:42.050168 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f263dbe-fba5-49f2-8d21-e9d657142bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:42.050444 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f263dbe-fba5-49f2-8d21-e9d657142bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:42.050712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f263dbe-fba5-49f2-8d21-e9d657142bb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:42.051427 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f263dbe-fba5-49f2-8d21-e9d657142bb4 None admin] RBAC: Adding query filter params (role.id=753f8b35d5f0458d8f62d0ad6e28033f, scope.project.id=c0e051a87db447378722164df3f66964, user.id=d4c2e44d11a74ced925a3d7e24aa92e5) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:42.051732 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f263dbe-fba5-49f2-8d21-e9d657142bb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:42.053674 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f263dbe-fba5-49f2-8d21-e9d657142bb4 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:42.064503 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 543/1061] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:11:41 2022] GET /identity/v3/role_assignments?user.id=d4c2e44d11a74ced925a3d7e24aa92e5&scope.project.id=c0e051a87db447378722164df3f66964&role.id=753f8b35d5f0458d8f62d0ad6e28033f => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:42.269165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c539546-319f-4c22-93ec-f531fcc032af None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:42.333273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:42.373346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c539546-319f-4c22-93ec-f531fcc032af None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:42.408304 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7eb04a-9266-420a-b6d9-861bc093ca4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:42.408891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7eb04a-9266-420a-b6d9-861bc093ca4c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:42.409176 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7eb04a-9266-420a-b6d9-861bc093ca4c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:42.409446 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7eb04a-9266-420a-b6d9-861bc093ca4c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:42.419565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7eb04a-9266-420a-b6d9-861bc093ca4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:11:42.424448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-5a7eb04a-9266-420a-b6d9-861bc093ca4c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:11:42.424448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:42.424448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:42.424448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:42.424448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:42.424448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:42.424448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:42.424448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:42.424448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:42.424448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:42.424448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:42.424448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:42.424448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:11:42.424448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:11:42.424448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:11:42.424448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:11:42.425368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:11:42.425368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:11:42.425368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:11:42.425368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:11:42.425368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:42.425368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:42.425368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:42.425368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:42.425368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:42.425368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:42.425368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:42.425368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:42.425368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:42.425368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:42.425368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:42.425368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:42.426210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:42.426210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:42.426210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:42.426210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:42.426210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:11:42.426210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:11:42.426210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:11:42.426210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:11:42.426210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:11:42.426210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:11:42.426210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:11:42.426210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:42.426210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 519/1062] 10.222.0.81 () {56 vars in 1120 bytes} [Mon Sep 19 11:11:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:42.432848 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f263dbe-fba5-49f2-8d21-e9d657142bb4 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:42.536920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f263dbe-fba5-49f2-8d21-e9d657142bb4 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:42.572024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b2da5-91c9-4bd4-8915-4e76db170a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:42.572622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b2da5-91c9-4bd4-8915-4e76db170a1b None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:42.572895 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b2da5-91c9-4bd4-8915-4e76db170a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:42.573160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b2da5-91c9-4bd4-8915-4e76db170a1b None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:42.573813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b2da5-91c9-4bd4-8915-4e76db170a1b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:42.574110 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b2da5-91c9-4bd4-8915-4e76db170a1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:42.575459 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b2da5-91c9-4bd4-8915-4e76db170a1b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:42.585477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 544/1063] 10.222.0.81 () {56 vars in 1115 bytes} [Mon Sep 19 11:11:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:42.590874 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7eb04a-9266-420a-b6d9-861bc093ca4c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:42.699261 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7eb04a-9266-420a-b6d9-861bc093ca4c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:42.728299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:42.749806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c37bfb-e6f2-49ed-8e87-6f45c2cfd087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:42.750434 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c37bfb-e6f2-49ed-8e87-6f45c2cfd087 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:42.750709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c37bfb-e6f2-49ed-8e87-6f45c2cfd087 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:42.750978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c37bfb-e6f2-49ed-8e87-6f45c2cfd087 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:42.761536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c37bfb-e6f2-49ed-8e87-6f45c2cfd087 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:42.762980 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c37bfb-e6f2-49ed-8e87-6f45c2cfd087 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:42.778639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-24c37bfb-e6f2-49ed-8e87-6f45c2cfd087 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:42.778639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:42.778639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:42.778639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:42.778639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:42.778639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:42.778639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:42.778639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:42.778639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:42.778639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:42.778639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:42.778639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:42.778639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:42.778639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:42.778639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:42.778639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:42.779609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:42.779609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:42.779609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:42.779609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:42.779609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:42.779609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:42.779609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:42.779609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:42.779609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:42.779609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:42.779609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:42.779609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:42.779609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:42.779609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:42.779609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:42.779609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:42.779609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:42.780912 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:42.780912 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:42.780912 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:42.785493 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b2da5-91c9-4bd4-8915-4e76db170a1b None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:42.785926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 520/1064] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:42.832907 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:42.897617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b2da5-91c9-4bd4-8915-4e76db170a1b None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:42.934194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b081da3-da74-4dcd-928f-2f4fefa12656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:42.934851 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b081da3-da74-4dcd-928f-2f4fefa12656 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:42.935135 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b081da3-da74-4dcd-928f-2f4fefa12656 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:42.935409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b081da3-da74-4dcd-928f-2f4fefa12656 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:42.936092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b081da3-da74-4dcd-928f-2f4fefa12656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:42.936399 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b081da3-da74-4dcd-928f-2f4fefa12656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:42.937751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b081da3-da74-4dcd-928f-2f4fefa12656 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:42.954437 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 545/1065] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:42.955553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c37bfb-e6f2-49ed-8e87-6f45c2cfd087 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:43.055614 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c37bfb-e6f2-49ed-8e87-6f45c2cfd087 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:43.089795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e4569-0b25-445e-ad50-069d21545ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:43.090422 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e4569-0b25-445e-ad50-069d21545ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:43.090704 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e4569-0b25-445e-ad50-069d21545ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:43.090981 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e4569-0b25-445e-ad50-069d21545ed6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:43.098547 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e4569-0b25-445e-ad50-069d21545ed6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:43.100127 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e4569-0b25-445e-ad50-069d21545ed6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:43.111210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-914e4569-0b25-445e-ad50-069d21545ed6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:11:43.111210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:43.111210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:43.111210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:43.111210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:43.111210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:43.111210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:43.111210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:43.111210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:43.111210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:43.111210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:43.111210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:43.111210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:11:43.111210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:11:43.111210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:11:43.111210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:11:43.112209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:43.112209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:43.112209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:11:43.112209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:43.112209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:11:43.112209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:43.112209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:43.112209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:43.112209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:43.112209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:43.112209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:43.112209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:43.112209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:43.112209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:43.112209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:43.112209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:43.112209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:43.112209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:43.117495 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:43.117495 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:43.117495 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:11:43.117495 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:11:43.117495 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:11:43.117495 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:11:43.117495 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:11:43.117495 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:11:43.117495 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:11:43.117495 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:43.117495 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 521/1066] 10.222.0.81 () {56 vars in 1093 bytes} [Mon Sep 19 11:11:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:43.120846 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b081da3-da74-4dcd-928f-2f4fefa12656 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:43.223746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:43.232421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b081da3-da74-4dcd-928f-2f4fefa12656 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:43.266322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258075a1-3478-44f0-90ed-7d927d6b4327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:43.266947 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258075a1-3478-44f0-90ed-7d927d6b4327 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:43.267223 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258075a1-3478-44f0-90ed-7d927d6b4327 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:43.267489 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258075a1-3478-44f0-90ed-7d927d6b4327 None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:43.268234 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258075a1-3478-44f0-90ed-7d927d6b4327 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:43.268536 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258075a1-3478-44f0-90ed-7d927d6b4327 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:43.270106 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258075a1-3478-44f0-90ed-7d927d6b4327 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:43.292341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 546/1067] 10.222.0.81 () {56 vars in 1133 bytes} [Mon Sep 19 11:11:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:43.293822 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e4569-0b25-445e-ad50-069d21545ed6 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:43.331247 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:43.399041 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e4569-0b25-445e-ad50-069d21545ed6 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:43.433883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6df79e-0c14-4f3e-bce7-bd830615a303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:43.434503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6df79e-0c14-4f3e-bce7-bd830615a303 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:43.434779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6df79e-0c14-4f3e-bce7-bd830615a303 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:43.435044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6df79e-0c14-4f3e-bce7-bd830615a303 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:43.445146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6df79e-0c14-4f3e-bce7-bd830615a303 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:43.446591 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6df79e-0c14-4f3e-bce7-bd830615a303 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:43.461142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-ab6df79e-0c14-4f3e-bce7-bd830615a303 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:43.461142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:43.461142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:43.461142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:43.461142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:43.461142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:43.461142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:43.461142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:43.461142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:43.461142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:43.461142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:43.461142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:43.461142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:43.461142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:43.461142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:43.461142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:43.462099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:43.462099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:43.462099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:43.462099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:43.462099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:43.462099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:43.462099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:43.462099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:43.462099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:43.462099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:43.462099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:43.462099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:43.462099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:43.462099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:43.462099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:43.462099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:43.462099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:43.463015 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:43.463015 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:43.463015 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:43.464078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 522/1068] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:43.467900 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258075a1-3478-44f0-90ed-7d927d6b4327 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:43.571087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258075a1-3478-44f0-90ed-7d927d6b4327 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:43.605068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c26642-ffe8-4612-a02e-2645933c101c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:43.605666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c26642-ffe8-4612-a02e-2645933c101c None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:43.605942 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c26642-ffe8-4612-a02e-2645933c101c None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:43.606210 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c26642-ffe8-4612-a02e-2645933c101c None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:43.606840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c26642-ffe8-4612-a02e-2645933c101c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:43.607137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c26642-ffe8-4612-a02e-2645933c101c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:43.608502 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c26642-ffe8-4612-a02e-2645933c101c None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:43.624503 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 547/1069] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:43.626032 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6df79e-0c14-4f3e-bce7-bd830615a303 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:43.726375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:43.739642 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6df79e-0c14-4f3e-bce7-bd830615a303 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:43.776332 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e1cdb-5544-40bb-ab5a-9b96b5ed4a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:43.776959 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e1cdb-5544-40bb-ab5a-9b96b5ed4a1a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:43.777227 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e1cdb-5544-40bb-ab5a-9b96b5ed4a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:43.777497 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e1cdb-5544-40bb-ab5a-9b96b5ed4a1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:43.784279 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e1cdb-5544-40bb-ab5a-9b96b5ed4a1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:43.785736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e1cdb-5544-40bb-ab5a-9b96b5ed4a1a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:43.797120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-bd3e1cdb-5544-40bb-ab5a-9b96b5ed4a1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:43.797120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:43.797120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:43.797120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:43.797120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:43.797120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:43.797120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:43.797120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:43.797120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:43.797120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:43.797120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:43.797120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:43.797120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:11:43.797120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:11:43.797120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:11:43.798039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:11:43.798039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:43.798039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:43.798039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:43.798039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:43.798039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:43.798039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:43.798039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:43.798039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:43.798039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:43.798039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:43.798039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:43.798039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:43.798039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:43.798039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:43.798039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:43.798039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:43.804469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:11:43.804469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:11:43.804469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:11:43.804469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:11:43.804469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:11:43.804469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:11:43.804469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:43.804469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:43.804469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 523/1070] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:11:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:43.804971 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c26642-ffe8-4612-a02e-2645933c101c None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:43.824839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:43.914365 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c26642-ffe8-4612-a02e-2645933c101c None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:43.950905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b87bbb-a1d2-4403-b2d7-5cd504b95a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:43.951505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b87bbb-a1d2-4403-b2d7-5cd504b95a4e None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:43.951794 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b87bbb-a1d2-4403-b2d7-5cd504b95a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:43.952069 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b87bbb-a1d2-4403-b2d7-5cd504b95a4e None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:43.953013 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b87bbb-a1d2-4403-b2d7-5cd504b95a4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:43.953312 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b87bbb-a1d2-4403-b2d7-5cd504b95a4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:43.954704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b87bbb-a1d2-4403-b2d7-5cd504b95a4e None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:43.966836 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 548/1071] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:11:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:43.972569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e1cdb-5544-40bb-ab5a-9b96b5ed4a1a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:44.029852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b5bf9-33ad-418e-81ec-adf655dacf85 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:44.030141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b5bf9-33ad-418e-81ec-adf655dacf85 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:44.030414 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b5bf9-33ad-418e-81ec-adf655dacf85 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:44.031742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 549/1072] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:11:44.036997 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d4463-1c7f-44df-87ae-0fb0b46deb71 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:44.038025 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d4463-1c7f-44df-87ae-0fb0b46deb71 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:44.038306 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d4463-1c7f-44df-87ae-0fb0b46deb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:44.058550 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-9a7d4463-1c7f-44df-87ae-0fb0b46deb71 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:44.059734 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-9a7d4463-1c7f-44df-87ae-0fb0b46deb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:44.078893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e1cdb-5544-40bb-ab5a-9b96b5ed4a1a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:44.116389 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f52a67-a95e-4bab-ab4f-aa6d3eff0682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:44.117053 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f52a67-a95e-4bab-ab4f-aa6d3eff0682 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:44.117341 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f52a67-a95e-4bab-ab4f-aa6d3eff0682 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:44.117610 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f52a67-a95e-4bab-ab4f-aa6d3eff0682 None admin] PATH_INFO: `/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/5abd70a18ccd407bafe3b61c8870986e/roles/7689e410cf25472397444b35809be2c7` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:44.119789 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f52a67-a95e-4bab-ab4f-aa6d3eff0682 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed9a0b626e3e4b93b391158e981ec586, user_id=5abd70a18ccd407bafe3b61c8870986e, role_id=7689e410cf25472397444b35809be2c7)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:44.121240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f52a67-a95e-4bab-ab4f-aa6d3eff0682 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:44.134823 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 550/1073] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:11:44.135978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 524/1074] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:11:43 2022] PUT /identity/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/5abd70a18ccd407bafe3b61c8870986e/roles/7689e410cf25472397444b35809be2c7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:11:44.201540 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439dc4e9-d774-47d9-873d-1f710ab931d7 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:44.201910 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439dc4e9-d774-47d9-873d-1f710ab931d7 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:44.202088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439dc4e9-d774-47d9-873d-1f710ab931d7 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:44.203792 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 551/1075] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:44.209336 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca98be1-6d80-4f8f-a3d2-5a3a7d298115 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:44.209636 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca98be1-6d80-4f8f-a3d2-5a3a7d298115 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:44.210055 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca98be1-6d80-4f8f-a3d2-5a3a7d298115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:44.230514 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-aca98be1-6d80-4f8f-a3d2-5a3a7d298115 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:44.231902 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-aca98be1-6d80-4f8f-a3d2-5a3a7d298115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:44.234253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:44.311291 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 525/1076] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:44.402191 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439dc4e9-d774-47d9-873d-1f710ab931d7 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:44.413057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:44.502503 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439dc4e9-d774-47d9-873d-1f710ab931d7 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:44.535304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f742a3a-4303-4a40-9e71-87e475adad5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:44.535926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f742a3a-4303-4a40-9e71-87e475adad5f None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:44.536200 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f742a3a-4303-4a40-9e71-87e475adad5f None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:44.536464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f742a3a-4303-4a40-9e71-87e475adad5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:44.601614 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f742a3a-4303-4a40-9e71-87e475adad5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:44.603150 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f742a3a-4303-4a40-9e71-87e475adad5f None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:44.605285 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca98be1-6d80-4f8f-a3d2-5a3a7d298115 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:44.614958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:44.715500 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca98be1-6d80-4f8f-a3d2-5a3a7d298115 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:44.717672 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 552/1077] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:11:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:11:44.723057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f742a3a-4303-4a40-9e71-87e475adad5f None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:44.755591 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af881cf-48c6-4d91-9fbe-e533dc82575f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:44.756781 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af881cf-48c6-4d91-9fbe-e533dc82575f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:44.757061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af881cf-48c6-4d91-9fbe-e533dc82575f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:44.757332 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af881cf-48c6-4d91-9fbe-e533dc82575f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:44.765067 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:44.769789 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af881cf-48c6-4d91-9fbe-e533dc82575f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:11:44.776590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-1af881cf-48c6-4d91-9fbe-e533dc82575f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:11:44.776590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:44.776590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:44.776590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:44.776590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:44.776590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:44.776590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:44.776590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:44.776590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:44.776590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:44.776590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:44.776590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:44.776590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:11:44.776590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:11:44.776590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:11:44.776590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:11:44.777600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:11:44.777600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:11:44.777600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:11:44.777600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:11:44.777600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:44.777600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:44.777600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:44.777600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:44.777600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:44.777600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:44.777600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:44.777600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:44.777600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:44.777600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:44.777600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:44.777600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:44.778595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:44.778595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:44.778595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:44.778595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:44.778595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:11:44.778595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:11:44.778595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:11:44.778595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:11:44.778595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:11:44.778595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:11:44.778595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:11:44.778595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:44.778595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 526/1078] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:11:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:44.783701 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af881cf-48c6-4d91-9fbe-e533dc82575f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:44.844769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f742a3a-4303-4a40-9e71-87e475adad5f None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:44.857856 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:44.890923 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de063f6-e1a0-4faa-aa9a-5773eb8aa2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:44.891843 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de063f6-e1a0-4faa-aa9a-5773eb8aa2aa None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:44.892121 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de063f6-e1a0-4faa-aa9a-5773eb8aa2aa None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:44.892530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de063f6-e1a0-4faa-aa9a-5773eb8aa2aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:44.895919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af881cf-48c6-4d91-9fbe-e533dc82575f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:44.905453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de063f6-e1a0-4faa-aa9a-5773eb8aa2aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:44.907257 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de063f6-e1a0-4faa-aa9a-5773eb8aa2aa None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:44.923297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-8de063f6-e1a0-4faa-aa9a-5773eb8aa2aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:44.923297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:44.923297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:44.923297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:44.923297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:44.923297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:44.923297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:44.923297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:44.923297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:44.923297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:44.923297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:44.923297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:44.923297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:44.923297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:44.923297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:44.923297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:44.924576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:44.924576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:44.924576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:44.924576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:44.924576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:44.924576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:44.924576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:44.924576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:44.924576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:44.924576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:44.924576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:44.924576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:44.924576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:44.924576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:44.924576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:44.924576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:44.924576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:44.926061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:44.926061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:44.926061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:44.929380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 553/1079] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:44.930063 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de063f6-e1a0-4faa-aa9a-5773eb8aa2aa None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:44.933790 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f1f12-d8c8-45e4-b354-3f18df616a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:44.934473 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f1f12-d8c8-45e4-b354-3f18df616a4c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:44.934757 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f1f12-d8c8-45e4-b354-3f18df616a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:44.935030 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f1f12-d8c8-45e4-b354-3f18df616a4c None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:44.935748 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f1f12-d8c8-45e4-b354-3f18df616a4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:44.936081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f1f12-d8c8-45e4-b354-3f18df616a4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:44.937478 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f1f12-d8c8-45e4-b354-3f18df616a4c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:44.947595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 527/1080] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:11:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:44.953285 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f1f12-d8c8-45e4-b354-3f18df616a4c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:45.019184 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:45.042164 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de063f6-e1a0-4faa-aa9a-5773eb8aa2aa None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:45.062354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f1f12-d8c8-45e4-b354-3f18df616a4c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:45.077222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb4c20-8da4-4bb5-ae3f-9282257895b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:45.078315 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb4c20-8da4-4bb5-ae3f-9282257895b6 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:45.078788 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb4c20-8da4-4bb5-ae3f-9282257895b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:45.079244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb4c20-8da4-4bb5-ae3f-9282257895b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:45.080093 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bb4c20-8da4-4bb5-ae3f-9282257895b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:45.080393 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bb4c20-8da4-4bb5-ae3f-9282257895b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:45.081745 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bb4c20-8da4-4bb5-ae3f-9282257895b6 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:45.095111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 554/1081] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:45.099189 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260afdb6-3d82-4d3b-810e-baa2ad44f7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:45.099936 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260afdb6-3d82-4d3b-810e-baa2ad44f7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:45.100187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260afdb6-3d82-4d3b-810e-baa2ad44f7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:45.100456 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260afdb6-3d82-4d3b-810e-baa2ad44f7c6 None admin] PATH_INFO: `/v3/users/d4c2e44d11a74ced925a3d7e24aa92e5` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:45.101658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260afdb6-3d82-4d3b-810e-baa2ad44f7c6 None admin] RBAC: Authorizing `identity:get_user(user_id=d4c2e44d11a74ced925a3d7e24aa92e5)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:45.103134 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260afdb6-3d82-4d3b-810e-baa2ad44f7c6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:45.105056 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 528/1082] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:11:44 2022] GET /identity/v3/users/d4c2e44d11a74ced925a3d7e24aa92e5 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:45.110832 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb4c20-8da4-4bb5-ae3f-9282257895b6 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:45.135118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:45.221605 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb4c20-8da4-4bb5-ae3f-9282257895b6 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:45.258398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ce130-f610-4477-ab3e-d878b24b2c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:45.259031 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ce130-f610-4477-ab3e-d878b24b2c8b None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:45.259308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ce130-f610-4477-ab3e-d878b24b2c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:45.259580 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ce130-f610-4477-ab3e-d878b24b2c8b None admin] PATH_INFO: `/v3/projects/c0e051a87db447378722164df3f66964` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:45.260741 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ce130-f610-4477-ab3e-d878b24b2c8b None admin] RBAC: Authorizing `identity:get_project(project_id=c0e051a87db447378722164df3f66964)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:45.262228 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ce130-f610-4477-ab3e-d878b24b2c8b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:45.267659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 555/1083] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:11:45 2022] GET /identity/v3/projects/c0e051a87db447378722164df3f66964 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:45.269801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260afdb6-3d82-4d3b-810e-baa2ad44f7c6 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:45.385045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260afdb6-3d82-4d3b-810e-baa2ad44f7c6 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:45.412609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:45.422003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017caba-e40d-44fe-adf7-0ed3f924bc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:45.422886 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017caba-e40d-44fe-adf7-0ed3f924bc30 None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:45.423327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017caba-e40d-44fe-adf7-0ed3f924bc30 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:45.423599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017caba-e40d-44fe-adf7-0ed3f924bc30 None admin] PATH_INFO: `/v3/projects/c0e051a87db447378722164df3f66964/users/d4c2e44d11a74ced925a3d7e24aa92e5/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:45.425522 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5017caba-e40d-44fe-adf7-0ed3f924bc30 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0e051a87db447378722164df3f66964, user_id=d4c2e44d11a74ced925a3d7e24aa92e5, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:45.426955 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5017caba-e40d-44fe-adf7-0ed3f924bc30 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:45.440940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 529/1084] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:11:45 2022] PUT /identity/v3/projects/c0e051a87db447378722164df3f66964/users/d4c2e44d11a74ced925a3d7e24aa92e5/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:11:46.281052 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc6d9d-e24d-4a43-800a-c169e7a17de8 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:46.281625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc6d9d-e24d-4a43-800a-c169e7a17de8 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:46.281625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc6d9d-e24d-4a43-800a-c169e7a17de8 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:46.282916 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 556/1085] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:46.288713 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee416c-e57b-4037-a4df-89e82ef76895 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:46.289180 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee416c-e57b-4037-a4df-89e82ef76895 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:46.289275 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee416c-e57b-4037-a4df-89e82ef76895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:46.309603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8bee416c-e57b-4037-a4df-89e82ef76895 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:46.310882 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8bee416c-e57b-4037-a4df-89e82ef76895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:46.377261 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 530/1086] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:46.647812 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc6d9d-e24d-4a43-800a-c169e7a17de8 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:46.737549 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:46.751450 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc6d9d-e24d-4a43-800a-c169e7a17de8 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:46.786809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfeea4-52b1-4230-81b4-335efc7cd875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:46.787477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfeea4-52b1-4230-81b4-335efc7cd875 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:46.787752 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfeea4-52b1-4230-81b4-335efc7cd875 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:46.788093 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfeea4-52b1-4230-81b4-335efc7cd875 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:46.797890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cfeea4-52b1-4230-81b4-335efc7cd875 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:11:46.802581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-54cfeea4-52b1-4230-81b4-335efc7cd875 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:11:46.802581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:46.802581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:46.802581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:46.802581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:46.802581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:46.802581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:46.802581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:46.802581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:46.802581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:46.802581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:46.802581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:46.802581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:11:46.802581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:11:46.802581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:11:46.802581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:11:46.804126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:11:46.804126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:11:46.804126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:11:46.804126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:11:46.804126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:46.804126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:46.804126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:46.804126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:46.804126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:46.804126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:46.804126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:46.804126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:46.804126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:46.804126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:46.804126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:46.804126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:46.805009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:46.805009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:46.805009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:46.805009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:46.805009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:11:46.805009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:11:46.805009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:11:46.805009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:11:46.805009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:11:46.805009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:11:46.805009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 11:11:46.805009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:46.805009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 557/1087] 10.222.0.81 () {56 vars in 1120 bytes} [Mon Sep 19 11:11:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:46.810288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee416c-e57b-4037-a4df-89e82ef76895 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:46.911496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee416c-e57b-4037-a4df-89e82ef76895 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:46.929223 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:46.946917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5c4cd-242d-49a4-a1a3-b0504b3f9437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:46.947519 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5c4cd-242d-49a4-a1a3-b0504b3f9437 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:46.947826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5c4cd-242d-49a4-a1a3-b0504b3f9437 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:46.948102 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5c4cd-242d-49a4-a1a3-b0504b3f9437 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:46.948767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc5c4cd-242d-49a4-a1a3-b0504b3f9437 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:46.949070 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc5c4cd-242d-49a4-a1a3-b0504b3f9437 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:46.950500 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc5c4cd-242d-49a4-a1a3-b0504b3f9437 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:46.958485 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 531/1088] 10.222.0.81 () {56 vars in 1115 bytes} [Mon Sep 19 11:11:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:46.964331 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfeea4-52b1-4230-81b4-335efc7cd875 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:47.061288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfeea4-52b1-4230-81b4-335efc7cd875 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:47.093530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcdc45-9e81-4a3e-b9ee-a8dab651b750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:47.094148 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcdc45-9e81-4a3e-b9ee-a8dab651b750 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:47.094427 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcdc45-9e81-4a3e-b9ee-a8dab651b750 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:47.094699 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcdc45-9e81-4a3e-b9ee-a8dab651b750 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:47.105445 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcdc45-9e81-4a3e-b9ee-a8dab651b750 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:47.106880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcdc45-9e81-4a3e-b9ee-a8dab651b750 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:47.114650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:47.121657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-60bcdc45-9e81-4a3e-b9ee-a8dab651b750 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:47.121657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:47.121657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:47.121657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:47.121657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:47.121657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:47.121657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:47.121657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:47.121657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:47.121657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:47.121657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:47.121657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:47.121657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:47.121657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:47.121657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:47.121657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:47.127588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:47.127588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:47.127588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:47.127588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:47.127588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:47.127588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:47.127588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:47.127588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:47.127588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:47.127588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:47.127588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:47.127588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:47.127588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:47.127588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:47.127588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:47.127588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:47.127588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:47.128697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:47.128697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:47.128697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:47.128697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 558/1089] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:47.128942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5c4cd-242d-49a4-a1a3-b0504b3f9437 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:47.227328 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5c4cd-242d-49a4-a1a3-b0504b3f9437 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:47.260219 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c52f0-8acf-4894-bd5e-2b654d35bb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:47.260822 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c52f0-8acf-4894-bd5e-2b654d35bb34 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:47.261104 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c52f0-8acf-4894-bd5e-2b654d35bb34 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:47.261380 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c52f0-8acf-4894-bd5e-2b654d35bb34 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:47.262022 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c52f0-8acf-4894-bd5e-2b654d35bb34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:47.262326 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c52f0-8acf-4894-bd5e-2b654d35bb34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:47.263694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c52f0-8acf-4894-bd5e-2b654d35bb34 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:47.276005 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 532/1090] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:47.281608 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcdc45-9e81-4a3e-b9ee-a8dab651b750 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:47.379546 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcdc45-9e81-4a3e-b9ee-a8dab651b750 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:47.413047 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707db8b-a4af-414d-a90b-a378a562a4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:47.413667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707db8b-a4af-414d-a90b-a378a562a4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:47.413946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707db8b-a4af-414d-a90b-a378a562a4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:47.414218 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707db8b-a4af-414d-a90b-a378a562a4c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:47.421507 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707db8b-a4af-414d-a90b-a378a562a4c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:47.422927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707db8b-a4af-414d-a90b-a378a562a4c7 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:47.433816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-7707db8b-a4af-414d-a90b-a378a562a4c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:11:47.433816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:47.433816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:47.433816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:47.433816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:47.433816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:47.433816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:47.433816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:47.433816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:47.433816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:47.433816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:47.433816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:47.433816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:11:47.433816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:11:47.433816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:11:47.433816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:11:47.434769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:47.434769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:47.434769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:11:47.434769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:47.434769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:11:47.434769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:11:47.434769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:47.434769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:47.434769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:47.434769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:47.434769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:47.434769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:47.434769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:47.434769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:47.434769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:47.434769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:47.434769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:47.434769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:47.436152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:47.436152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:47.436152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:11:47.436152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:11:47.436152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:11:47.436152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:11:47.436152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:11:47.436152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:11:47.436152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 11:11:47.436152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:47.436152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 559/1091] 10.222.0.81 () {56 vars in 1093 bytes} [Mon Sep 19 11:11:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:47.440619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c52f0-8acf-4894-bd5e-2b654d35bb34 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:47.485598 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:47.544326 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c52f0-8acf-4894-bd5e-2b654d35bb34 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:47.576981 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163b79a-95dd-4aa8-ac85-bbdb8c088aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:47.577834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163b79a-95dd-4aa8-ac85-bbdb8c088aeb None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:47.578113 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163b79a-95dd-4aa8-ac85-bbdb8c088aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:47.578380 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163b79a-95dd-4aa8-ac85-bbdb8c088aeb None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:47.579102 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163b79a-95dd-4aa8-ac85-bbdb8c088aeb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:47.579404 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163b79a-95dd-4aa8-ac85-bbdb8c088aeb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:47.580859 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163b79a-95dd-4aa8-ac85-bbdb8c088aeb None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:47.585167 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dae8d6-a626-4048-a56b-8ffa76b24a0a None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:47.585458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dae8d6-a626-4048-a56b-8ffa76b24a0a None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:47.585725 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dae8d6-a626-4048-a56b-8ffa76b24a0a None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:47.587171 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 560/1092] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:47.591871 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d4ec9-c815-4f3a-a437-47a61d4426c5 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:47.592152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d4ec9-c815-4f3a-a437-47a61d4426c5 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:47.592421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d4ec9-c815-4f3a-a437-47a61d4426c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:47.603799 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 533/1093] 10.222.0.81 () {56 vars in 1133 bytes} [Mon Sep 19 11:11:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:47.604892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163b79a-95dd-4aa8-ac85-bbdb8c088aeb None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:47.610545 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-b28d4ec9-c815-4f3a-a437-47a61d4426c5 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:47.611723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b28d4ec9-c815-4f3a-a437-47a61d4426c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:47.667190 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:47.681913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 561/1094] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:11:47.700894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:47.704940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163b79a-95dd-4aa8-ac85-bbdb8c088aeb None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:47.738356 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a6bb7-89f3-4821-a932-2ff54fad65e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:47.738920 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a6bb7-89f3-4821-a932-2ff54fad65e0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:47.739199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a6bb7-89f3-4821-a932-2ff54fad65e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:47.739469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a6bb7-89f3-4821-a932-2ff54fad65e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:47.804735 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a6bb7-89f3-4821-a932-2ff54fad65e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:47.806179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a6bb7-89f3-4821-a932-2ff54fad65e0 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:47.898451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:47.903939 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 534/1095] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:11:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 11:11:47.909169 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d4ec9-c815-4f3a-a437-47a61d4426c5 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:47.964916 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a6bb7-89f3-4821-a932-2ff54fad65e0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:48.007685 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d4ec9-c815-4f3a-a437-47a61d4426c5 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:48.041067 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3c044-763e-4ae2-b8da-cc98b4a4e1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:48.041691 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3c044-763e-4ae2-b8da-cc98b4a4e1e4 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:48.041968 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3c044-763e-4ae2-b8da-cc98b4a4e1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:48.042239 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3c044-763e-4ae2-b8da-cc98b4a4e1e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:48.052442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3c044-763e-4ae2-b8da-cc98b4a4e1e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:48.054368 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3c044-763e-4ae2-b8da-cc98b4a4e1e4 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:48.063985 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a6bb7-89f3-4821-a932-2ff54fad65e0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:48.068825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-08f3c044-763e-4ae2-b8da-cc98b4a4e1e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:48.068825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:48.068825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:48.068825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:48.068825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:48.068825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:48.068825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:48.068825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:48.068825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:48.068825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:48.068825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:48.068825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:48.068825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:11:48.068825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:11:48.068825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:11:48.068825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:11:48.069772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:48.069772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:48.069772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:48.069772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:48.069772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:48.069772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:48.069772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:48.069772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:48.069772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:48.069772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:48.069772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:48.069772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:48.069772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:48.069772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:48.069772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:48.069772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:48.069772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:11:48.070697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:11:48.070697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:11:48.070697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:48.070697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 562/1096] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:11:48.075863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3c044-763e-4ae2-b8da-cc98b4a4e1e4 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:48.098604 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2ca70-6f06-4888-9b7b-3c0720d2d2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:48.099264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2ca70-6f06-4888-9b7b-3c0720d2d2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:48.099541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2ca70-6f06-4888-9b7b-3c0720d2d2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:48.099958 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2ca70-6f06-4888-9b7b-3c0720d2d2b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:48.110424 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2ca70-6f06-4888-9b7b-3c0720d2d2b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:11:48.115217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-6db2ca70-6f06-4888-9b7b-3c0720d2d2b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:11:48.115217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:48.115217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:48.115217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:48.115217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:48.115217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:48.115217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:48.115217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:48.115217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:48.115217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:48.115217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:48.115217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:48.115217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:11:48.115217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:11:48.115217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:11:48.115217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:11:48.116821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:11:48.116821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:11:48.116821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:11:48.116821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:11:48.116821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:48.116821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:48.116821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:48.116821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:48.116821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:48.116821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:48.116821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:48.116821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:48.116821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:48.116821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:48.116821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:48.116821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:48.117684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:48.117684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:48.117684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:48.117684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:48.117684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:11:48.117684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:11:48.117684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:11:48.117684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:11:48.117684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:11:48.117684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:11:48.117684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:11:48.117684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:11:48.117684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 535/1097] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:11:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:48.121459 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:48.122829 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2ca70-6f06-4888-9b7b-3c0720d2d2b0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:48.179167 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3c044-763e-4ae2-b8da-cc98b4a4e1e4 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:48.182222 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:48.215253 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f215e-5df9-4396-ba8b-9cde5e877c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:48.215900 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f215e-5df9-4396-ba8b-9cde5e877c29 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:48.218439 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f215e-5df9-4396-ba8b-9cde5e877c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:48.218730 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f215e-5df9-4396-ba8b-9cde5e877c29 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:48.219391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f215e-5df9-4396-ba8b-9cde5e877c29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:48.219719 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f215e-5df9-4396-ba8b-9cde5e877c29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:48.221099 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f215e-5df9-4396-ba8b-9cde5e877c29 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:48.229070 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2ca70-6f06-4888-9b7b-3c0720d2d2b0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:48.233743 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 563/1098] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:48.239300 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f215e-5df9-4396-ba8b-9cde5e877c29 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:48.266661 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389a495-fe31-453a-bf41-57df3ae38c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:48.267200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389a495-fe31-453a-bf41-57df3ae38c4d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:48.267476 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389a495-fe31-453a-bf41-57df3ae38c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:48.267751 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389a495-fe31-453a-bf41-57df3ae38c4d None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:48.268443 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389a495-fe31-453a-bf41-57df3ae38c4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:48.268773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389a495-fe31-453a-bf41-57df3ae38c4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:48.270409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389a495-fe31-453a-bf41-57df3ae38c4d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:48.281939 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 536/1099] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:11:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:48.287490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389a495-fe31-453a-bf41-57df3ae38c4d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:48.341591 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f215e-5df9-4396-ba8b-9cde5e877c29 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:48.343288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:48.377520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2386ad-2ac9-4fd7-8616-a66ec2ad1a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:48.378139 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2386ad-2ac9-4fd7-8616-a66ec2ad1a8b None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:48.378423 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2386ad-2ac9-4fd7-8616-a66ec2ad1a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:48.378696 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2386ad-2ac9-4fd7-8616-a66ec2ad1a8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:48.385173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389a495-fe31-453a-bf41-57df3ae38c4d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:48.385823 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2386ad-2ac9-4fd7-8616-a66ec2ad1a8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:48.387244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2386ad-2ac9-4fd7-8616-a66ec2ad1a8b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:48.398198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-2b2386ad-2ac9-4fd7-8616-a66ec2ad1a8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:48.398198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:11:48.398198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:11:48.398198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:11:48.398198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:11:48.398198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:11:48.398198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:11:48.398198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:11:48.398198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:11:48.398198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:11:48.398198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:11:48.398198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:11:48.398198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:11:48.398198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:11:48.398198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:11:48.399098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:11:48.399098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:11:48.399098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:11:48.399098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:11:48.399098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:11:48.399098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:11:48.399098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:11:48.399098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:11:48.399098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:11:48.399098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:11:48.399098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:11:48.399098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:11:48.399098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:11:48.399098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:11:48.399098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:11:48.399098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:11:48.399098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:11:48.400061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:11:48.400061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:11:48.400061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:11:48.400061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:11:48.400061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:11:48.400061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:11:48.400061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:11:48.400061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:11:48.400061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 564/1100] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:11:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:11:48.404854 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:48.404974 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2386ad-2ac9-4fd7-8616-a66ec2ad1a8b None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:48.420737 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f44c2b-3da9-4206-9970-bd523a45080d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:48.421370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f44c2b-3da9-4206-9970-bd523a45080d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:48.421649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f44c2b-3da9-4206-9970-bd523a45080d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:48.421920 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f44c2b-3da9-4206-9970-bd523a45080d None admin] PATH_INFO: `/v3/users/d4c2e44d11a74ced925a3d7e24aa92e5` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:48.423057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f44c2b-3da9-4206-9970-bd523a45080d None admin] RBAC: Authorizing `identity:get_user(user_id=d4c2e44d11a74ced925a3d7e24aa92e5)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:48.424545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f44c2b-3da9-4206-9970-bd523a45080d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:48.426123 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 537/1101] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:11:48 2022] GET /identity/v3/users/d4c2e44d11a74ced925a3d7e24aa92e5 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:48.431734 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f44c2b-3da9-4206-9970-bd523a45080d None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:48.503187 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2386ad-2ac9-4fd7-8616-a66ec2ad1a8b None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:48.528810 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f44c2b-3da9-4206-9970-bd523a45080d None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:48.536229 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567465f0-fe11-4961-a019-e0292e19c799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:48.536815 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567465f0-fe11-4961-a019-e0292e19c799 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:48.537106 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567465f0-fe11-4961-a019-e0292e19c799 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:48.537383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567465f0-fe11-4961-a019-e0292e19c799 None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:48.538059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567465f0-fe11-4961-a019-e0292e19c799 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:48.538361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567465f0-fe11-4961-a019-e0292e19c799 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:48.539791 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567465f0-fe11-4961-a019-e0292e19c799 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:48.551353 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 565/1102] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:11:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:48.557262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567465f0-fe11-4961-a019-e0292e19c799 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:48.562475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69868af4-57ac-4963-aee8-6b5eab10a3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:48.563084 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69868af4-57ac-4963-aee8-6b5eab10a3aa None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:48.563364 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69868af4-57ac-4963-aee8-6b5eab10a3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:48.563638 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69868af4-57ac-4963-aee8-6b5eab10a3aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:48.581245 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:48.622153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:48.634795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69868af4-57ac-4963-aee8-6b5eab10a3aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:48.636281 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69868af4-57ac-4963-aee8-6b5eab10a3aa None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:48.664250 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567465f0-fe11-4961-a019-e0292e19c799 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:48.698990 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca85e0f-6a02-4909-b369-be396b39c20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:48.699568 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca85e0f-6a02-4909-b369-be396b39c20a None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:48.699873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca85e0f-6a02-4909-b369-be396b39c20a None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:48.700148 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca85e0f-6a02-4909-b369-be396b39c20a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:48.700863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca85e0f-6a02-4909-b369-be396b39c20a None admin] RBAC: Adding query filter params (role.id=7689e410cf25472397444b35809be2c7, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=5abd70a18ccd407bafe3b61c8870986e) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:48.701167 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca85e0f-6a02-4909-b369-be396b39c20a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:48.702826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca85e0f-6a02-4909-b369-be396b39c20a None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:48.708428 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 566/1103] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:11:48 2022] GET /identity/v3/role_assignments?user.id=5abd70a18ccd407bafe3b61c8870986e&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=7689e410cf25472397444b35809be2c7 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:48.736074 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 538/1104] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:11:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:11:48.742798 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca85e0f-6a02-4909-b369-be396b39c20a None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:48.834498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:48.851041 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca85e0f-6a02-4909-b369-be396b39c20a None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:48.889376 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986594a2-99b1-48c2-9b7f-2642dea567b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:48.890014 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986594a2-99b1-48c2-9b7f-2642dea567b2 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:48.890288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986594a2-99b1-48c2-9b7f-2642dea567b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:48.890558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986594a2-99b1-48c2-9b7f-2642dea567b2 None admin] PATH_INFO: `/v3/projects/c0e051a87db447378722164df3f66964` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:48.891674 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986594a2-99b1-48c2-9b7f-2642dea567b2 None admin] RBAC: Authorizing `identity:get_project(project_id=c0e051a87db447378722164df3f66964)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:48.893202 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986594a2-99b1-48c2-9b7f-2642dea567b2 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:48.894977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 567/1105] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:11:48 2022] GET /identity/v3/projects/c0e051a87db447378722164df3f66964 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:48.901670 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69868af4-57ac-4963-aee8-6b5eab10a3aa None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:48.993946 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:49.006132 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69868af4-57ac-4963-aee8-6b5eab10a3aa None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:49.042364 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3249730-f626-4a10-8b82-74d55762997b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:49.042969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3249730-f626-4a10-8b82-74d55762997b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:49.043249 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3249730-f626-4a10-8b82-74d55762997b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:49.043523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3249730-f626-4a10-8b82-74d55762997b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:49.044274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3249730-f626-4a10-8b82-74d55762997b None admin] RBAC: Adding query filter params (role.id=753f8b35d5f0458d8f62d0ad6e28033f, scope.project.id=c0e051a87db447378722164df3f66964, user.id=d4c2e44d11a74ced925a3d7e24aa92e5) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:49.044585 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3249730-f626-4a10-8b82-74d55762997b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:49.046047 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3249730-f626-4a10-8b82-74d55762997b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:49.052144 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 539/1106] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:11:48 2022] GET /identity/v3/role_assignments?user.id=d4c2e44d11a74ced925a3d7e24aa92e5&scope.project.id=c0e051a87db447378722164df3f66964&role.id=753f8b35d5f0458d8f62d0ad6e28033f => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:11:51.215380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93639abb-d192-4b58-a077-1d7c50264fa1 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:51.215979 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93639abb-d192-4b58-a077-1d7c50264fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:51.215979 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93639abb-d192-4b58-a077-1d7c50264fa1 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:51.217279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 568/1107] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:51.222350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d048247-d7dc-402c-b477-8717c0d38ba4 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:51.222792 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d048247-d7dc-402c-b477-8717c0d38ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:51.222887 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d048247-d7dc-402c-b477-8717c0d38ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:51.241106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8d048247-d7dc-402c-b477-8717c0d38ba4 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:51.242275 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8d048247-d7dc-402c-b477-8717c0d38ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:51.308089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 540/1108] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:51.587312 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93639abb-d192-4b58-a077-1d7c50264fa1 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:51.685314 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93639abb-d192-4b58-a077-1d7c50264fa1 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:51.716848 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d6824-7aad-45e1-9572-bc8cb845af89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:51.717456 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d6824-7aad-45e1-9572-bc8cb845af89 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:51.717728 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d6824-7aad-45e1-9572-bc8cb845af89 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:51.717989 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d6824-7aad-45e1-9572-bc8cb845af89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:51.719071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d6824-7aad-45e1-9572-bc8cb845af89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:51.720499 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d6824-7aad-45e1-9572-bc8cb845af89 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:51.722020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 569/1109] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:11:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:51.728194 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d048247-d7dc-402c-b477-8717c0d38ba4 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:51.807346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:51.825226 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d048247-d7dc-402c-b477-8717c0d38ba4 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:51.859018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c9cca-86e8-4ad7-a110-c00c504098a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:51.859617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c9cca-86e8-4ad7-a110-c00c504098a8 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:51.859925 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c9cca-86e8-4ad7-a110-c00c504098a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:51.860207 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c9cca-86e8-4ad7-a110-c00c504098a8 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:51.860852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031c9cca-86e8-4ad7-a110-c00c504098a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:51.862282 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031c9cca-86e8-4ad7-a110-c00c504098a8 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:51.882660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 541/1110] 10.222.0.81 () {60 vars in 1132 bytes} [Mon Sep 19 11:11:51 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:11:54.055967 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22014b-7059-4ed5-944a-b05289ebd4ee None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:54.056474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22014b-7059-4ed5-944a-b05289ebd4ee None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:54.056474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22014b-7059-4ed5-944a-b05289ebd4ee None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:54.057751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 570/1111] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:54.063050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbbceb-f3e4-4bfe-8d8d-f08d17a60222 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:54.063471 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbbceb-f3e4-4bfe-8d8d-f08d17a60222 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:54.063580 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbbceb-f3e4-4bfe-8d8d-f08d17a60222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:54.082141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-3cfbbceb-f3e4-4bfe-8d8d-f08d17a60222 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:54.083326 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3cfbbceb-f3e4-4bfe-8d8d-f08d17a60222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:54.146815 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 542/1112] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:54.420464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22014b-7059-4ed5-944a-b05289ebd4ee None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:54.444861 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:54.518815 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22014b-7059-4ed5-944a-b05289ebd4ee None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:54.550413 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa892d-a68a-414c-9e3e-5c7cab863092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:54.551018 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa892d-a68a-414c-9e3e-5c7cab863092 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:54.551291 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa892d-a68a-414c-9e3e-5c7cab863092 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:54.551562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa892d-a68a-414c-9e3e-5c7cab863092 None admin] PATH_INFO: `/v3/roles/4cc230d31ac94f4c89bf827491c01bd7` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:54.552952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa892d-a68a-414c-9e3e-5c7cab863092 None admin] RBAC: Authorizing `identity:get_role(role_id=4cc230d31ac94f4c89bf827491c01bd7)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:54.554292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa892d-a68a-414c-9e3e-5c7cab863092 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:54.555560 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 571/1113] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:11:54 2022] GET /identity/v3/roles/4cc230d31ac94f4c89bf827491c01bd7 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:54.561118 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfbbceb-f3e4-4bfe-8d8d-f08d17a60222 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:54.624300 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:54.658178 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfbbceb-f3e4-4bfe-8d8d-f08d17a60222 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:54.692241 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7496d5-c0ec-4964-811c-6b06047b9a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:54.692865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7496d5-c0ec-4964-811c-6b06047b9a6a None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:54.693686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7496d5-c0ec-4964-811c-6b06047b9a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:54.693959 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7496d5-c0ec-4964-811c-6b06047b9a6a None admin] PATH_INFO: `/v3/users/06f2c0c0d32440bb839260a88d9a4440` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:54.706974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7496d5-c0ec-4964-811c-6b06047b9a6a None admin] RBAC: Authorizing `identity:get_user(user_id=06f2c0c0d32440bb839260a88d9a4440)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:54.708438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7496d5-c0ec-4964-811c-6b06047b9a6a None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:54.710641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 543/1114] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:11:54 2022] GET /identity/v3/users/06f2c0c0d32440bb839260a88d9a4440 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:54.716355 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa892d-a68a-414c-9e3e-5c7cab863092 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:54.803882 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:54.813665 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa892d-a68a-414c-9e3e-5c7cab863092 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:54.846429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a5d73-bf95-45ea-a0a6-0371cf9693fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:54.847016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a5d73-bf95-45ea-a0a6-0371cf9693fa None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:54.847290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a5d73-bf95-45ea-a0a6-0371cf9693fa None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:54.847571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a5d73-bf95-45ea-a0a6-0371cf9693fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:54.910726 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a5d73-bf95-45ea-a0a6-0371cf9693fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:54.912199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a5d73-bf95-45ea-a0a6-0371cf9693fa None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:54.998058 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:55.008453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 572/1115] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:11:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:11:55.014335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7496d5-c0ec-4964-811c-6b06047b9a6a None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:55.112421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7496d5-c0ec-4964-811c-6b06047b9a6a None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:55.136834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:55.145692 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74853b2-7dc9-4387-9861-09f588e4626c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:55.146835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74853b2-7dc9-4387-9861-09f588e4626c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:55.147110 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74853b2-7dc9-4387-9861-09f588e4626c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:55.147380 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74853b2-7dc9-4387-9861-09f588e4626c None admin] PATH_INFO: `/v3/projects/c0e051a87db447378722164df3f66964` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:55.148523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74853b2-7dc9-4387-9861-09f588e4626c None admin] RBAC: Authorizing `identity:get_project(project_id=c0e051a87db447378722164df3f66964)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:55.149944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74853b2-7dc9-4387-9861-09f588e4626c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:55.151572 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 544/1116] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:11:55 2022] GET /identity/v3/projects/c0e051a87db447378722164df3f66964 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:55.157675 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a5d73-bf95-45ea-a0a6-0371cf9693fa None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:55.253073 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a5d73-bf95-45ea-a0a6-0371cf9693fa None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:55.286003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a4927-0b5a-4e92-a83a-f0f5f12eb5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:55.286588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a4927-0b5a-4e92-a83a-f0f5f12eb5e4 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:55.286893 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a4927-0b5a-4e92-a83a-f0f5f12eb5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:55.287167 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a4927-0b5a-4e92-a83a-f0f5f12eb5e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:55.287905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a4927-0b5a-4e92-a83a-f0f5f12eb5e4 None admin] RBAC: Adding query filter params (role.id=4cc230d31ac94f4c89bf827491c01bd7, scope.project.id=c0e051a87db447378722164df3f66964, user.id=06f2c0c0d32440bb839260a88d9a4440) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:11:55.288218 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a4927-0b5a-4e92-a83a-f0f5f12eb5e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:55.289620 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a4927-0b5a-4e92-a83a-f0f5f12eb5e4 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:55.294794 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 573/1117] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:11:55 2022] GET /identity/v3/role_assignments?user.id=06f2c0c0d32440bb839260a88d9a4440&scope.project.id=c0e051a87db447378722164df3f66964&role.id=4cc230d31ac94f4c89bf827491c01bd7 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:57.386485 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b0261-5ae7-454a-8f06-fe05eb83bf1b None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:57.387059 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b0261-5ae7-454a-8f06-fe05eb83bf1b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:57.387059 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b0261-5ae7-454a-8f06-fe05eb83bf1b None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:57.388406 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 545/1118] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:11:57.393414 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6ece7-faa7-4301-80b6-a6bca773b99e None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:57.393805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6ece7-faa7-4301-80b6-a6bca773b99e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:57.393951 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6ece7-faa7-4301-80b6-a6bca773b99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:57.412832 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-d2a6ece7-faa7-4301-80b6-a6bca773b99e None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:11:57.414014 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-d2a6ece7-faa7-4301-80b6-a6bca773b99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:11:57.461236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:57.478268 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 574/1119] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:11:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:11:57.749953 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b0261-5ae7-454a-8f06-fe05eb83bf1b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:57.848511 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b0261-5ae7-454a-8f06-fe05eb83bf1b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:57.881103 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73818c7f-6bb0-49fc-8930-1a6ceb125782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:57.881713 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73818c7f-6bb0-49fc-8930-1a6ceb125782 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:57.881994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73818c7f-6bb0-49fc-8930-1a6ceb125782 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:57.882265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73818c7f-6bb0-49fc-8930-1a6ceb125782 None admin] PATH_INFO: `/v3/roles/4cc230d31ac94f4c89bf827491c01bd7` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:57.883645 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73818c7f-6bb0-49fc-8930-1a6ceb125782 None admin] RBAC: Authorizing `identity:get_role(role_id=4cc230d31ac94f4c89bf827491c01bd7)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:57.885042 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73818c7f-6bb0-49fc-8930-1a6ceb125782 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:57.886344 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 546/1120] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:11:57 2022] GET /identity/v3/roles/4cc230d31ac94f4c89bf827491c01bd7 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:57.892064 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6ece7-faa7-4301-80b6-a6bca773b99e None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:57.986594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6ece7-faa7-4301-80b6-a6bca773b99e None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:58.018360 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8cb3f6-4ff8-40c6-8a68-a154f2ae78b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:58.018966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cb3f6-4ff8-40c6-8a68-a154f2ae78b2 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:58.019243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cb3f6-4ff8-40c6-8a68-a154f2ae78b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:58.019515 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cb3f6-4ff8-40c6-8a68-a154f2ae78b2 None admin] PATH_INFO: `/v3/users/06f2c0c0d32440bb839260a88d9a4440` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:58.020637 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8cb3f6-4ff8-40c6-8a68-a154f2ae78b2 None admin] RBAC: Authorizing `identity:get_user(user_id=06f2c0c0d32440bb839260a88d9a4440)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:58.022541 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8cb3f6-4ff8-40c6-8a68-a154f2ae78b2 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:58.024107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 575/1121] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:11:57 2022] GET /identity/v3/users/06f2c0c0d32440bb839260a88d9a4440 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:58.029699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73818c7f-6bb0-49fc-8930-1a6ceb125782 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:58.089878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:58.127408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73818c7f-6bb0-49fc-8930-1a6ceb125782 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:58.160342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941876b-f59b-4d5f-8a9d-4a9c54a9c217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:58.160956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941876b-f59b-4d5f-8a9d-4a9c54a9c217 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:58.161234 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941876b-f59b-4d5f-8a9d-4a9c54a9c217 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:58.161505 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941876b-f59b-4d5f-8a9d-4a9c54a9c217 None admin] PATH_INFO: `/v3/projects/c0e051a87db447378722164df3f66964` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:58.163168 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941876b-f59b-4d5f-8a9d-4a9c54a9c217 None admin] RBAC: Authorizing `identity:get_project(project_id=c0e051a87db447378722164df3f66964)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:58.164629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941876b-f59b-4d5f-8a9d-4a9c54a9c217 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:58.166243 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 547/1122] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:11:58 2022] GET /identity/v3/projects/c0e051a87db447378722164df3f66964 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:11:58.172279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8cb3f6-4ff8-40c6-8a68-a154f2ae78b2 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:11:58.223657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:11:58.268977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8cb3f6-4ff8-40c6-8a68-a154f2ae78b2 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:11:58.301815 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797405cd-5c3b-4544-8106-5158b3507ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:11:58.303011 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797405cd-5c3b-4544-8106-5158b3507ade None admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:11:58.303281 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797405cd-5c3b-4544-8106-5158b3507ade None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:11:58.303546 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797405cd-5c3b-4544-8106-5158b3507ade None admin] PATH_INFO: `/v3/projects/c0e051a87db447378722164df3f66964/users/06f2c0c0d32440bb839260a88d9a4440/roles/4cc230d31ac94f4c89bf827491c01bd7` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:11:58.305666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797405cd-5c3b-4544-8106-5158b3507ade None admin] RBAC: Authorizing `identity:create_grant(project_id=c0e051a87db447378722164df3f66964, user_id=06f2c0c0d32440bb839260a88d9a4440, role_id=4cc230d31ac94f4c89bf827491c01bd7)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:11:58.307070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797405cd-5c3b-4544-8106-5158b3507ade None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:11:58.319075 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 576/1123] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:11:58 2022] PUT /identity/v3/projects/c0e051a87db447378722164df3f66964/users/06f2c0c0d32440bb839260a88d9a4440/roles/4cc230d31ac94f4c89bf827491c01bd7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:12:00.466940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973732ea-c027-4912-b598-e1623b858459 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:00.467489 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973732ea-c027-4912-b598-e1623b858459 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:00.467815 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973732ea-c027-4912-b598-e1623b858459 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:00.469181 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 548/1124] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:12:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:12:00.474535 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c5eb6-a0f5-48ae-b2d4-0d2dd695f40c None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:00.474991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c5eb6-a0f5-48ae-b2d4-0d2dd695f40c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:00.475100 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c5eb6-a0f5-48ae-b2d4-0d2dd695f40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:00.492749 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-0f2c5eb6-a0f5-48ae-b2d4-0d2dd695f40c None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:12:00.493933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-0f2c5eb6-a0f5-48ae-b2d4-0d2dd695f40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:12:00.564271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 577/1125] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:12:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:12:00.830829 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973732ea-c027-4912-b598-e1623b858459 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:00.929701 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973732ea-c027-4912-b598-e1623b858459 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:00.957748 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:00.963887 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06fdfdf-4336-46e6-be24-e7a4852d65bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:00.964505 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fdfdf-4336-46e6-be24-e7a4852d65bf None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:00.964784 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fdfdf-4336-46e6-be24-e7a4852d65bf None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:00.965057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fdfdf-4336-46e6-be24-e7a4852d65bf None admin] PATH_INFO: `/v3/roles/4cc230d31ac94f4c89bf827491c01bd7` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:00.966438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06fdfdf-4336-46e6-be24-e7a4852d65bf None admin] RBAC: Authorizing `identity:get_role(role_id=4cc230d31ac94f4c89bf827491c01bd7)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:00.967845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06fdfdf-4336-46e6-be24-e7a4852d65bf None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:00.969154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 549/1126] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:12:00 2022] GET /identity/v3/roles/4cc230d31ac94f4c89bf827491c01bd7 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:00.974845 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c5eb6-a0f5-48ae-b2d4-0d2dd695f40c None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:01.016270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:01.077125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c5eb6-a0f5-48ae-b2d4-0d2dd695f40c None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:01.109693 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf5815-ce92-48a1-944d-712638dd090b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:01.110314 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf5815-ce92-48a1-944d-712638dd090b None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:01.110594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf5815-ce92-48a1-944d-712638dd090b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:01.110869 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf5815-ce92-48a1-944d-712638dd090b None admin] PATH_INFO: `/v3/users/06f2c0c0d32440bb839260a88d9a4440` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:01.111996 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf5815-ce92-48a1-944d-712638dd090b None admin] RBAC: Authorizing `identity:get_user(user_id=06f2c0c0d32440bb839260a88d9a4440)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:01.113425 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf5815-ce92-48a1-944d-712638dd090b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:01.115673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 578/1127] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:12:00 2022] GET /identity/v3/users/06f2c0c0d32440bb839260a88d9a4440 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:01.121425 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06fdfdf-4336-46e6-be24-e7a4852d65bf None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:01.218510 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06fdfdf-4336-46e6-be24-e7a4852d65bf None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:01.251551 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8626f1-7a7f-4fff-91cc-605f29e3eb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:01.252167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8626f1-7a7f-4fff-91cc-605f29e3eb88 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:01.252450 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8626f1-7a7f-4fff-91cc-605f29e3eb88 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:01.252723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8626f1-7a7f-4fff-91cc-605f29e3eb88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:01.316414 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:01.320530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8626f1-7a7f-4fff-91cc-605f29e3eb88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:01.321969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8626f1-7a7f-4fff-91cc-605f29e3eb88 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:01.415105 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 550/1128] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:12:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:12:01.420922 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf5815-ce92-48a1-944d-712638dd090b None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:01.516932 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf5815-ce92-48a1-944d-712638dd090b None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:01.527658 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:01.550947 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4381e5d-9946-404e-a2f8-e93cff1cfcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:01.551560 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4381e5d-9946-404e-a2f8-e93cff1cfcd1 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:01.551874 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4381e5d-9946-404e-a2f8-e93cff1cfcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:01.552214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4381e5d-9946-404e-a2f8-e93cff1cfcd1 None admin] PATH_INFO: `/v3/projects/c0e051a87db447378722164df3f66964` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:01.553351 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4381e5d-9946-404e-a2f8-e93cff1cfcd1 None admin] RBAC: Authorizing `identity:get_project(project_id=c0e051a87db447378722164df3f66964)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:01.554849 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4381e5d-9946-404e-a2f8-e93cff1cfcd1 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:01.557975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 579/1129] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:12:01 2022] GET /identity/v3/projects/c0e051a87db447378722164df3f66964 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:01.565337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8626f1-7a7f-4fff-91cc-605f29e3eb88 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:01.658435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:01.662431 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8626f1-7a7f-4fff-91cc-605f29e3eb88 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:01.696564 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd5eff-c4d8-47ec-ada0-8b27bbe84d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:01.697158 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd5eff-c4d8-47ec-ada0-8b27bbe84d2d None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:01.697439 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd5eff-c4d8-47ec-ada0-8b27bbe84d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:01.697710 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd5eff-c4d8-47ec-ada0-8b27bbe84d2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:01.698434 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd5eff-c4d8-47ec-ada0-8b27bbe84d2d None admin] RBAC: Adding query filter params (role.id=4cc230d31ac94f4c89bf827491c01bd7, scope.project.id=c0e051a87db447378722164df3f66964, user.id=06f2c0c0d32440bb839260a88d9a4440) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:12:01.698744 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd5eff-c4d8-47ec-ada0-8b27bbe84d2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:01.700206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd5eff-c4d8-47ec-ada0-8b27bbe84d2d None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:01.706041 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 551/1130] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:12:01 2022] GET /identity/v3/role_assignments?user.id=06f2c0c0d32440bb839260a88d9a4440&scope.project.id=c0e051a87db447378722164df3f66964&role.id=4cc230d31ac94f4c89bf827491c01bd7 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:03.866804 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdcd9d-8ceb-4e75-b997-cbe445427b27 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:03.867337 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdcd9d-8ceb-4e75-b997-cbe445427b27 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:03.867337 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdcd9d-8ceb-4e75-b997-cbe445427b27 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:03.868677 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 580/1131] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:12:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:12:03.874088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf18d3-b1a6-41ce-9f2a-0f2149f1a339 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:03.874589 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf18d3-b1a6-41ce-9f2a-0f2149f1a339 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:03.874689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf18d3-b1a6-41ce-9f2a-0f2149f1a339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:03.896977 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-cddf18d3-b1a6-41ce-9f2a-0f2149f1a339 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:12:03.898177 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cddf18d3-b1a6-41ce-9f2a-0f2149f1a339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:12:03.963713 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 552/1132] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:12:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:12:04.231035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bdcd9d-8ceb-4e75-b997-cbe445427b27 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:04.331625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bdcd9d-8ceb-4e75-b997-cbe445427b27 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:04.365311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f4893-9c40-4cfa-893b-a22de6eb4776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:04.365916 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f4893-9c40-4cfa-893b-a22de6eb4776 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:04.366189 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f4893-9c40-4cfa-893b-a22de6eb4776 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:04.366453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f4893-9c40-4cfa-893b-a22de6eb4776 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:04.367554 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f4893-9c40-4cfa-893b-a22de6eb4776 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:04.369024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f4893-9c40-4cfa-893b-a22de6eb4776 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:04.370570 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 581/1133] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:12:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:04.376393 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf18d3-b1a6-41ce-9f2a-0f2149f1a339 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:04.462830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:04.471436 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf18d3-b1a6-41ce-9f2a-0f2149f1a339 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:04.504748 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6648d2-20d1-452e-b715-59c61cbc9efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:04.505308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6648d2-20d1-452e-b715-59c61cbc9efc None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:04.505586 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6648d2-20d1-452e-b715-59c61cbc9efc None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:04.505859 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6648d2-20d1-452e-b715-59c61cbc9efc None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:04.506515 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6648d2-20d1-452e-b715-59c61cbc9efc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:04.507961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6648d2-20d1-452e-b715-59c61cbc9efc None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:04.534517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 553/1134] 10.222.0.81 () {60 vars in 1141 bytes} [Mon Sep 19 11:12:04 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:12:06.681079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478832c7-bbc3-493a-adcb-2d7210638feb None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:06.681538 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478832c7-bbc3-493a-adcb-2d7210638feb None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:06.681662 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478832c7-bbc3-493a-adcb-2d7210638feb None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:06.682927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 582/1135] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:12:06.688299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc99697-0c00-447e-b47c-f1fd2f871c5f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:06.688725 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc99697-0c00-447e-b47c-f1fd2f871c5f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:06.688853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc99697-0c00-447e-b47c-f1fd2f871c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:06.707213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-1bc99697-0c00-447e-b47c-f1fd2f871c5f None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:12:06.708515 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1bc99697-0c00-447e-b47c-f1fd2f871c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:12:06.773787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 554/1136] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:12:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:12:07.039642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478832c7-bbc3-493a-adcb-2d7210638feb None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:07.086535 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:07.140933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478832c7-bbc3-493a-adcb-2d7210638feb None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:07.173758 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d5efd-ff24-4323-b741-bef0e4c7abdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:07.174372 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d5efd-ff24-4323-b741-bef0e4c7abdc None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:07.174651 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d5efd-ff24-4323-b741-bef0e4c7abdc None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:07.174923 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d5efd-ff24-4323-b741-bef0e4c7abdc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:07.176050 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d5efd-ff24-4323-b741-bef0e4c7abdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:07.178065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d5efd-ff24-4323-b741-bef0e4c7abdc None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:07.179617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 583/1137] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:12:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:07.185297 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc99697-0c00-447e-b47c-f1fd2f871c5f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:07.257580 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:07.284228 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc99697-0c00-447e-b47c-f1fd2f871c5f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:07.318059 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9efa0a-5719-46c6-89ff-61e88faa5bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:07.318596 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9efa0a-5719-46c6-89ff-61e88faa5bc6 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:07.318873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9efa0a-5719-46c6-89ff-61e88faa5bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:07.319142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9efa0a-5719-46c6-89ff-61e88faa5bc6 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:07.319818 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9efa0a-5719-46c6-89ff-61e88faa5bc6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:07.321249 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9efa0a-5719-46c6-89ff-61e88faa5bc6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:07.341946 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 555/1138] 10.222.0.81 () {60 vars in 1132 bytes} [Mon Sep 19 11:12:07 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:12:09.489030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43938011-5a4c-4e16-a0cf-ed3bcd4c9acc None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:09.489575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43938011-5a4c-4e16-a0cf-ed3bcd4c9acc None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:09.489575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43938011-5a4c-4e16-a0cf-ed3bcd4c9acc None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:09.490879 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 584/1139] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:12:09.496038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ba330-0b1a-484c-a557-88b92c3bc825 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:09.496419 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ba330-0b1a-484c-a557-88b92c3bc825 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:09.496582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ba330-0b1a-484c-a557-88b92c3bc825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:09.514773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c95ba330-0b1a-484c-a557-88b92c3bc825 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:12:09.515967 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c95ba330-0b1a-484c-a557-88b92c3bc825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:12:09.579625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 556/1140] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:12:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:12:09.857933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43938011-5a4c-4e16-a0cf-ed3bcd4c9acc None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:09.955317 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43938011-5a4c-4e16-a0cf-ed3bcd4c9acc None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:09.968570 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:09.987751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e79b18-7582-43a9-82e8-f1b5d0be1a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:09.988378 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e79b18-7582-43a9-82e8-f1b5d0be1a23 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:09.988659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e79b18-7582-43a9-82e8-f1b5d0be1a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:09.988921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e79b18-7582-43a9-82e8-f1b5d0be1a23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:09.998972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e79b18-7582-43a9-82e8-f1b5d0be1a23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:12:10.003667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-c3e79b18-7582-43a9-82e8-f1b5d0be1a23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:12:10.003667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:12:10.003667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:12:10.003667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:12:10.003667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:12:10.003667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:12:10.003667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:12:10.003667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:12:10.003667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:12:10.003667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:12:10.003667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:12:10.003667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:12:10.003667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:12:10.003667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:12:10.003667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:12:10.003667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:12:10.005323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:12:10.005323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:12:10.005323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:12:10.005323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:12:10.005323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:12:10.005323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:12:10.005323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:12:10.005323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:12:10.005323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:12:10.005323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:12:10.005323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:12:10.005323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:12:10.005323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:12:10.005323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:12:10.005323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:12:10.005323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:12:10.006185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:12:10.006185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:12:10.006185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:12:10.006185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:12:10.006185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:12:10.006185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:12:10.006185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:12:10.006185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:12:10.006185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:12:10.006185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:12:10.006185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:12:10.006185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:12:10.006185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 585/1141] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:12:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:12:10.011175 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ba330-0b1a-484c-a557-88b92c3bc825 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:10.066847 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:10.107556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ba330-0b1a-484c-a557-88b92c3bc825 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:10.139538 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce6156-46bb-4438-aeb0-da9264231557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:10.140096 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce6156-46bb-4438-aeb0-da9264231557 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:10.140373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce6156-46bb-4438-aeb0-da9264231557 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:10.140642 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce6156-46bb-4438-aeb0-da9264231557 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:10.141305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce6156-46bb-4438-aeb0-da9264231557 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:12:10.141609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce6156-46bb-4438-aeb0-da9264231557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:10.142968 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce6156-46bb-4438-aeb0-da9264231557 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:10.151200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 557/1142] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:12:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:10.157003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e79b18-7582-43a9-82e8-f1b5d0be1a23 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:10.252325 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e79b18-7582-43a9-82e8-f1b5d0be1a23 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:10.286678 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c1b79-70cc-4b1a-8d38-e989b01da95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:10.287300 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c1b79-70cc-4b1a-8d38-e989b01da95f None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:10.287579 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c1b79-70cc-4b1a-8d38-e989b01da95f None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:10.287883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c1b79-70cc-4b1a-8d38-e989b01da95f None admin] PATH_INFO: `/v3/users/3ccbf3eba59b45b59a5c490754a9d03d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:10.300678 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c1b79-70cc-4b1a-8d38-e989b01da95f None admin] RBAC: Authorizing `identity:get_user(user_id=3ccbf3eba59b45b59a5c490754a9d03d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:10.302726 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c1b79-70cc-4b1a-8d38-e989b01da95f None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:10.304339 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 586/1143] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:12:10 2022] GET /identity/v3/users/3ccbf3eba59b45b59a5c490754a9d03d => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:10.310109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce6156-46bb-4438-aeb0-da9264231557 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:10.405929 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce6156-46bb-4438-aeb0-da9264231557 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:10.428038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:10.440205 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229ab76-68eb-408d-9d91-79ffde424d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:10.440806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229ab76-68eb-408d-9d91-79ffde424d8f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:10.441084 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229ab76-68eb-408d-9d91-79ffde424d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:10.441591 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229ab76-68eb-408d-9d91-79ffde424d8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:10.507330 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0229ab76-68eb-408d-9d91-79ffde424d8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:10.508799 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0229ab76-68eb-408d-9d91-79ffde424d8f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:10.604101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 558/1144] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:12:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:12:10.610046 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c1b79-70cc-4b1a-8d38-e989b01da95f None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:10.639689 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:10.707532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c1b79-70cc-4b1a-8d38-e989b01da95f None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:10.740544 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0734a-4fc5-4d9b-b620-84d01accded8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:10.741157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0734a-4fc5-4d9b-b620-84d01accded8 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:10.741435 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0734a-4fc5-4d9b-b620-84d01accded8 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:10.741714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0734a-4fc5-4d9b-b620-84d01accded8 None admin] PATH_INFO: `/v3/projects/832f7774ed384964aefdfee7fd305618` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:10.742811 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f0734a-4fc5-4d9b-b620-84d01accded8 None admin] RBAC: Authorizing `identity:get_project(project_id=832f7774ed384964aefdfee7fd305618)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:10.744256 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f0734a-4fc5-4d9b-b620-84d01accded8 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:10.745861 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 587/1145] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:12:10 2022] GET /identity/v3/projects/832f7774ed384964aefdfee7fd305618 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:10.751978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229ab76-68eb-408d-9d91-79ffde424d8f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:10.775236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:10.849083 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229ab76-68eb-408d-9d91-79ffde424d8f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:10.881166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d751d8-4a47-4992-8ccf-1e668fe24597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:10.881752 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d751d8-4a47-4992-8ccf-1e668fe24597 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:10.882027 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d751d8-4a47-4992-8ccf-1e668fe24597 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:10.882293 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d751d8-4a47-4992-8ccf-1e668fe24597 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:10.882997 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d751d8-4a47-4992-8ccf-1e668fe24597 None admin] RBAC: Adding query filter params (role.id=753f8b35d5f0458d8f62d0ad6e28033f, scope.project.id=832f7774ed384964aefdfee7fd305618, user.id=3ccbf3eba59b45b59a5c490754a9d03d) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:12:10.883298 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d751d8-4a47-4992-8ccf-1e668fe24597 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:10.884732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d751d8-4a47-4992-8ccf-1e668fe24597 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:10.889893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 559/1146] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:12:10 2022] GET /identity/v3/role_assignments?user.id=3ccbf3eba59b45b59a5c490754a9d03d&scope.project.id=832f7774ed384964aefdfee7fd305618&role.id=753f8b35d5f0458d8f62d0ad6e28033f => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:12.979779 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a84c83-8766-4139-87a2-653a54485276 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:12.980306 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a84c83-8766-4139-87a2-653a54485276 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:12.980306 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a84c83-8766-4139-87a2-653a54485276 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:12.981577 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 588/1147] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:12:12.986854 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4c2ab-a630-4d40-808d-7f64b70831a9 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:12.987237 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4c2ab-a630-4d40-808d-7f64b70831a9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:12.987397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4c2ab-a630-4d40-808d-7f64b70831a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:13.005661 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-55b4c2ab-a630-4d40-808d-7f64b70831a9 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:12:13.006836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-55b4c2ab-a630-4d40-808d-7f64b70831a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:12:13.069871 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 560/1148] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:12:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:12:13.335665 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a84c83-8766-4139-87a2-653a54485276 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:13.431638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:13.436179 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a84c83-8766-4139-87a2-653a54485276 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:13.468759 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20a056-8c29-4d87-956f-2649fe0499b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:13.469374 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20a056-8c29-4d87-956f-2649fe0499b4 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:13.469652 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20a056-8c29-4d87-956f-2649fe0499b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:13.469925 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20a056-8c29-4d87-956f-2649fe0499b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:13.480207 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20a056-8c29-4d87-956f-2649fe0499b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:12:13.484977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-cd20a056-8c29-4d87-956f-2649fe0499b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:12:13.484977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:12:13.484977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:12:13.484977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:12:13.484977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:12:13.484977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:12:13.484977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:12:13.484977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:12:13.484977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:12:13.484977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:12:13.484977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:12:13.484977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:12:13.484977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:12:13.484977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:12:13.484977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:12:13.484977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:12:13.486077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:12:13.486077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:12:13.486077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:12:13.486077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:12:13.486077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:12:13.486077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:12:13.486077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:12:13.486077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:12:13.486077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:12:13.486077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:12:13.486077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:12:13.486077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:12:13.486077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:12:13.486077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:12:13.486077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:12:13.486077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:12:13.487016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:12:13.487016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:12:13.487016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:12:13.487016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:12:13.487016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:12:13.487016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:12:13.487016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:12:13.487016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:12:13.487016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:12:13.487016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:12:13.487016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:12:13.487016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:12:13.487016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 589/1149] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:12:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:12:13.492850 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4c2ab-a630-4d40-808d-7f64b70831a9 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:13.512517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:13.591369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4c2ab-a630-4d40-808d-7f64b70831a9 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:13.623672 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d331d9-e100-4592-b6c0-3cda421126a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:13.624288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d331d9-e100-4592-b6c0-3cda421126a0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:13.624564 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d331d9-e100-4592-b6c0-3cda421126a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:13.624831 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d331d9-e100-4592-b6c0-3cda421126a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:13.625484 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d331d9-e100-4592-b6c0-3cda421126a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:12:13.625782 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d331d9-e100-4592-b6c0-3cda421126a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:13.627137 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d331d9-e100-4592-b6c0-3cda421126a0 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:13.634785 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 561/1150] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:12:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:13.640568 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20a056-8c29-4d87-956f-2649fe0499b4 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:13.737571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20a056-8c29-4d87-956f-2649fe0499b4 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:13.769554 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71b87f-3145-405c-b094-caa8a04179e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:13.770652 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71b87f-3145-405c-b094-caa8a04179e8 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:13.770930 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71b87f-3145-405c-b094-caa8a04179e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:13.771203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71b87f-3145-405c-b094-caa8a04179e8 None admin] PATH_INFO: `/v3/users/3ccbf3eba59b45b59a5c490754a9d03d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:13.772336 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71b87f-3145-405c-b094-caa8a04179e8 None admin] RBAC: Authorizing `identity:get_user(user_id=3ccbf3eba59b45b59a5c490754a9d03d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:13.773754 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71b87f-3145-405c-b094-caa8a04179e8 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:13.775311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 590/1151] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:12:13 2022] GET /identity/v3/users/3ccbf3eba59b45b59a5c490754a9d03d => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:13.781079 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d331d9-e100-4592-b6c0-3cda421126a0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:13.857935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:13.879193 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d331d9-e100-4592-b6c0-3cda421126a0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:13.912176 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cec1cd-f8a0-45d3-a37e-5dd75504a00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:13.912847 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cec1cd-f8a0-45d3-a37e-5dd75504a00e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:13.913131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cec1cd-f8a0-45d3-a37e-5dd75504a00e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:13.913404 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cec1cd-f8a0-45d3-a37e-5dd75504a00e None admin] PATH_INFO: `/v3/projects/832f7774ed384964aefdfee7fd305618` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:13.914523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cec1cd-f8a0-45d3-a37e-5dd75504a00e None admin] RBAC: Authorizing `identity:get_project(project_id=832f7774ed384964aefdfee7fd305618)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:13.915975 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cec1cd-f8a0-45d3-a37e-5dd75504a00e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:13.917580 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 562/1152] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:12:13 2022] GET /identity/v3/projects/832f7774ed384964aefdfee7fd305618 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:13.923405 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71b87f-3145-405c-b094-caa8a04179e8 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:13.951093 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:14.022486 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71b87f-3145-405c-b094-caa8a04179e8 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:14.055166 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f165173-1363-4e10-b2c8-85837e23c4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:14.055826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f165173-1363-4e10-b2c8-85837e23c4e2 None admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:14.056109 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f165173-1363-4e10-b2c8-85837e23c4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:14.056382 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f165173-1363-4e10-b2c8-85837e23c4e2 None admin] PATH_INFO: `/v3/projects/832f7774ed384964aefdfee7fd305618/users/3ccbf3eba59b45b59a5c490754a9d03d/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:14.058264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f165173-1363-4e10-b2c8-85837e23c4e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=832f7774ed384964aefdfee7fd305618, user_id=3ccbf3eba59b45b59a5c490754a9d03d, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:14.059676 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f165173-1363-4e10-b2c8-85837e23c4e2 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:14.072448 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 591/1153] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:12:13 2022] PUT /identity/v3/projects/832f7774ed384964aefdfee7fd305618/users/3ccbf3eba59b45b59a5c490754a9d03d/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:12:16.169311 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f0606-5eb1-4299-8ada-17f5f2ddc2c9 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:16.169942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f0606-5eb1-4299-8ada-17f5f2ddc2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:16.169942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f0606-5eb1-4299-8ada-17f5f2ddc2c9 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:16.171216 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 563/1154] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:12:16.176889 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae502076-1ed3-4b0d-873e-c048a05e4106 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:16.177295 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae502076-1ed3-4b0d-873e-c048a05e4106 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:16.177428 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae502076-1ed3-4b0d-873e-c048a05e4106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:16.195015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-ae502076-1ed3-4b0d-873e-c048a05e4106 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:12:16.196220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ae502076-1ed3-4b0d-873e-c048a05e4106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:12:16.259862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 592/1155] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:12:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:12:16.533296 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f0606-5eb1-4299-8ada-17f5f2ddc2c9 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:16.632699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f0606-5eb1-4299-8ada-17f5f2ddc2c9 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:16.666341 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca75b43-2a91-402f-8d4d-5614a32d5125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:16.666970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca75b43-2a91-402f-8d4d-5614a32d5125 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:16.667253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca75b43-2a91-402f-8d4d-5614a32d5125 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:16.667524 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca75b43-2a91-402f-8d4d-5614a32d5125 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:16.677964 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca75b43-2a91-402f-8d4d-5614a32d5125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:12:16.682773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-5ca75b43-2a91-402f-8d4d-5614a32d5125 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:12:16.682773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:12:16.682773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:12:16.682773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:12:16.682773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:12:16.682773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:12:16.682773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:12:16.682773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:12:16.682773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:12:16.682773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:12:16.682773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:12:16.682773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:12:16.682773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:12:16.682773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:12:16.682773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:12:16.682773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:12:16.683960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:12:16.683960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:12:16.683960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:12:16.683960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:12:16.683960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:12:16.683960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:12:16.683960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:12:16.683960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:12:16.683960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:12:16.683960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:12:16.683960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:12:16.683960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:12:16.683960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:12:16.683960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:12:16.683960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:12:16.683960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:12:16.685153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:12:16.685153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:12:16.685153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:12:16.685153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:12:16.685153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:12:16.685153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:12:16.685153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:12:16.685153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:12:16.685153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:12:16.685153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:12:16.685153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:12:16.685153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:12:16.685153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 564/1156] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:12:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:12:16.689976 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae502076-1ed3-4b0d-873e-c048a05e4106 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:16.708387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:16.787868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae502076-1ed3-4b0d-873e-c048a05e4106 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:16.819997 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372d6e9-fbb1-4e18-befc-9135ea5f3e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:16.820591 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372d6e9-fbb1-4e18-befc-9135ea5f3e16 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:16.820871 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372d6e9-fbb1-4e18-befc-9135ea5f3e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:16.821146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372d6e9-fbb1-4e18-befc-9135ea5f3e16 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:16.821799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372d6e9-fbb1-4e18-befc-9135ea5f3e16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:12:16.822101 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372d6e9-fbb1-4e18-befc-9135ea5f3e16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:16.823451 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372d6e9-fbb1-4e18-befc-9135ea5f3e16 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:16.831403 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 593/1157] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:12:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:16.837750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca75b43-2a91-402f-8d4d-5614a32d5125 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:16.846882 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:16.936007 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca75b43-2a91-402f-8d4d-5614a32d5125 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:16.969406 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680cedc-c5e3-4ae0-8791-ed91aac55486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:16.970018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680cedc-c5e3-4ae0-8791-ed91aac55486 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:16.970298 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680cedc-c5e3-4ae0-8791-ed91aac55486 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:16.970573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680cedc-c5e3-4ae0-8791-ed91aac55486 None admin] PATH_INFO: `/v3/users/3ccbf3eba59b45b59a5c490754a9d03d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:16.971695 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680cedc-c5e3-4ae0-8791-ed91aac55486 None admin] RBAC: Authorizing `identity:get_user(user_id=3ccbf3eba59b45b59a5c490754a9d03d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:16.973202 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680cedc-c5e3-4ae0-8791-ed91aac55486 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:16.974764 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 565/1158] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:12:16 2022] GET /identity/v3/users/3ccbf3eba59b45b59a5c490754a9d03d => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:16.980438 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372d6e9-fbb1-4e18-befc-9135ea5f3e16 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:17.060055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:17.079652 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372d6e9-fbb1-4e18-befc-9135ea5f3e16 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:17.111715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5668ab-3196-4c6d-9eab-7cf341c7b958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:17.112321 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5668ab-3196-4c6d-9eab-7cf341c7b958 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:17.112598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5668ab-3196-4c6d-9eab-7cf341c7b958 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:17.112864 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5668ab-3196-4c6d-9eab-7cf341c7b958 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:17.176467 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5668ab-3196-4c6d-9eab-7cf341c7b958 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:17.177889 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5668ab-3196-4c6d-9eab-7cf341c7b958 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:17.252554 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:17.274371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 594/1159] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:12:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:12:17.279966 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680cedc-c5e3-4ae0-8791-ed91aac55486 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:17.353840 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:17.378858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680cedc-c5e3-4ae0-8791-ed91aac55486 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:17.411836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e7901-5c86-4efb-a0f7-7d0b9bbe14d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:17.412445 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e7901-5c86-4efb-a0f7-7d0b9bbe14d8 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:17.412718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e7901-5c86-4efb-a0f7-7d0b9bbe14d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:17.412986 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e7901-5c86-4efb-a0f7-7d0b9bbe14d8 None admin] PATH_INFO: `/v3/projects/832f7774ed384964aefdfee7fd305618` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:17.414087 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e7901-5c86-4efb-a0f7-7d0b9bbe14d8 None admin] RBAC: Authorizing `identity:get_project(project_id=832f7774ed384964aefdfee7fd305618)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:17.415512 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e7901-5c86-4efb-a0f7-7d0b9bbe14d8 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:17.417146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 566/1160] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:12:17 2022] GET /identity/v3/projects/832f7774ed384964aefdfee7fd305618 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:17.423205 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5668ab-3196-4c6d-9eab-7cf341c7b958 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:17.517598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5668ab-3196-4c6d-9eab-7cf341c7b958 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:17.550491 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d7daf-d181-49e7-80ee-74ddfa7e5661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:17.551092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d7daf-d181-49e7-80ee-74ddfa7e5661 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:17.551372 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d7daf-d181-49e7-80ee-74ddfa7e5661 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:17.551643 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d7daf-d181-49e7-80ee-74ddfa7e5661 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:17.552388 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d7daf-d181-49e7-80ee-74ddfa7e5661 None admin] RBAC: Adding query filter params (role.id=753f8b35d5f0458d8f62d0ad6e28033f, scope.project.id=832f7774ed384964aefdfee7fd305618, user.id=3ccbf3eba59b45b59a5c490754a9d03d) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:12:17.552696 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d7daf-d181-49e7-80ee-74ddfa7e5661 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:17.554095 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d7daf-d181-49e7-80ee-74ddfa7e5661 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:17.559572 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 595/1161] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:12:17 2022] GET /identity/v3/role_assignments?user.id=3ccbf3eba59b45b59a5c490754a9d03d&scope.project.id=832f7774ed384964aefdfee7fd305618&role.id=753f8b35d5f0458d8f62d0ad6e28033f => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:19.719078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ac3c6-55cb-4f72-9796-aa92ae9f21d0 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:19.719554 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ac3c6-55cb-4f72-9796-aa92ae9f21d0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:19.719659 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ac3c6-55cb-4f72-9796-aa92ae9f21d0 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:19.720998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 567/1162] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:12:19.726362 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e1f14-3c96-49cf-863f-ec820fb1b589 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:19.726786 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e1f14-3c96-49cf-863f-ec820fb1b589 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:19.726889 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e1f14-3c96-49cf-863f-ec820fb1b589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:19.745764 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-bc3e1f14-3c96-49cf-863f-ec820fb1b589 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:12:19.746949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-bc3e1f14-3c96-49cf-863f-ec820fb1b589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:12:19.788781 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:19.817252 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 596/1163] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:12:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:12:20.084328 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ac3c6-55cb-4f72-9796-aa92ae9f21d0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:20.185689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ac3c6-55cb-4f72-9796-aa92ae9f21d0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:20.219704 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab95b7-67c4-4267-8b42-9c00fc4bd9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:20.220325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab95b7-67c4-4267-8b42-9c00fc4bd9d0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:20.220604 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab95b7-67c4-4267-8b42-9c00fc4bd9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:20.220874 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab95b7-67c4-4267-8b42-9c00fc4bd9d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:20.232260 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:20.287372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab95b7-67c4-4267-8b42-9c00fc4bd9d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:20.288846 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab95b7-67c4-4267-8b42-9c00fc4bd9d0 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:20.384313 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 568/1164] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:12:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:12:20.390241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e1f14-3c96-49cf-863f-ec820fb1b589 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:20.485017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e1f14-3c96-49cf-863f-ec820fb1b589 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:20.518243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99bfd4c-a444-4478-b9cc-f2831fd598a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:20.518857 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bfd4c-a444-4478-b9cc-f2831fd598a8 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:20.519136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bfd4c-a444-4478-b9cc-f2831fd598a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:20.519408 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bfd4c-a444-4478-b9cc-f2831fd598a8 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:20.526355 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99bfd4c-a444-4478-b9cc-f2831fd598a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:20.527814 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99bfd4c-a444-4478-b9cc-f2831fd598a8 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:20.539409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-d99bfd4c-a444-4478-b9cc-f2831fd598a8 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 19 11:12:20.539409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:12:20.539409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 19 11:12:20.539409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 19 11:12:20.539409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:12:20.539409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:12:20.539409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:12:20.539409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:12:20.539409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 19 11:12:20.539409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:12:20.539409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:12:20.539409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:12:20.539409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:12:20.539409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:12:20.539409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:12:20.539409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:12:20.539409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:12:20.540623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:12:20.540623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:12:20.540623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:12:20.540623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:12:20.540623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:12:20.540623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:12:20.540623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:12:20.540623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:12:20.540623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:12:20.540623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:12:20.540623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:12:20.540623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:12:20.540623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:12:20.540623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:12:20.540623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:12:20.540623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:12:20.541700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:12:20.541700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:12:20.541700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:12:20.541700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:12:20.541700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:12:20.541700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:12:20.541700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:12:20.541700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:12:20.541700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:12:20.541700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:12:20.541700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 19 11:12:20.541700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:12:20.541700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 19 11:12:20.541700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:12:20.541700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 597/1165] 10.222.0.81 () {56 vars in 1117 bytes} [Mon Sep 19 11:12:20 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:12:20.546575 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab95b7-67c4-4267-8b42-9c00fc4bd9d0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:20.588978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:20.644336 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab95b7-67c4-4267-8b42-9c00fc4bd9d0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:20.678808 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74fec1-83c9-41f3-939b-478aa9d04eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:20.679411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74fec1-83c9-41f3-939b-478aa9d04eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:20.679694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74fec1-83c9-41f3-939b-478aa9d04eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:20.680001 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74fec1-83c9-41f3-939b-478aa9d04eb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:20.680638 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74fec1-83c9-41f3-939b-478aa9d04eb0 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:12:20.680941 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74fec1-83c9-41f3-939b-478aa9d04eb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:20.682314 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74fec1-83c9-41f3-939b-478aa9d04eb0 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:20.689306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 569/1166] 10.222.0.81 () {56 vars in 1115 bytes} [Mon Sep 19 11:12:20 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:22.797717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8f311-b0b0-43a7-bbfe-7b1c7d775778 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:22.798248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8f311-b0b0-43a7-bbfe-7b1c7d775778 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:22.798248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8f311-b0b0-43a7-bbfe-7b1c7d775778 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:22.799574 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 598/1167] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:12:22.805192 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224221d-e306-4c9d-9e43-7db80e051ec5 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:22.805620 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224221d-e306-4c9d-9e43-7db80e051ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:22.805722 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224221d-e306-4c9d-9e43-7db80e051ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:22.823936 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e224221d-e306-4c9d-9e43-7db80e051ec5 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:12:22.825111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e224221d-e306-4c9d-9e43-7db80e051ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:12:22.888772 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 570/1168] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:12:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:12:23.155471 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8f311-b0b0-43a7-bbfe-7b1c7d775778 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:23.190399 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:23.256172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8f311-b0b0-43a7-bbfe-7b1c7d775778 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:23.289078 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99653a07-087a-4f35-9049-74201a8d79e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:23.289667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99653a07-087a-4f35-9049-74201a8d79e1 None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:23.289947 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99653a07-087a-4f35-9049-74201a8d79e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:23.290219 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99653a07-087a-4f35-9049-74201a8d79e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:23.290902 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99653a07-087a-4f35-9049-74201a8d79e1 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:23.292288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99653a07-087a-4f35-9049-74201a8d79e1 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:23.313107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 599/1169] 10.222.0.81 () {60 vars in 1138 bytes} [Mon Sep 19 11:12:23 2022] POST /identity/v3/domains => generated 267 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:12:25.473856 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f0889-ac01-4fcf-a573-796f7750466b None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:25.474427 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f0889-ac01-4fcf-a573-796f7750466b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:25.474427 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f0889-ac01-4fcf-a573-796f7750466b None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:25.475724 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 571/1170] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:12:25.480963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e05528-0f73-49e2-8b36-b09b493a3031 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:25.481356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e05528-0f73-49e2-8b36-b09b493a3031 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:25.481513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e05528-0f73-49e2-8b36-b09b493a3031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:25.499588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-30e05528-0f73-49e2-8b36-b09b493a3031 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:12:25.500803 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-30e05528-0f73-49e2-8b36-b09b493a3031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:12:25.565221 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 600/1171] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:12:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:12:25.833863 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f0889-ac01-4fcf-a573-796f7750466b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:25.869158 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:25.932839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f0889-ac01-4fcf-a573-796f7750466b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:25.966708 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e610cea-b321-4e8e-84e4-d02e3ba52f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:25.967314 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e610cea-b321-4e8e-84e4-d02e3ba52f68 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:25.967593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e610cea-b321-4e8e-84e4-d02e3ba52f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:25.967891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e610cea-b321-4e8e-84e4-d02e3ba52f68 None admin] PATH_INFO: `/v3/domains/c63acf42aaad40ceb6ce9125f0310c78` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:25.977809 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e610cea-b321-4e8e-84e4-d02e3ba52f68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c63acf42aaad40ceb6ce9125f0310c78)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:25.979249 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e610cea-b321-4e8e-84e4-d02e3ba52f68 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:25.980824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 572/1172] 10.222.0.81 () {56 vars in 1183 bytes} [Mon Sep 19 11:12:25 2022] GET /identity/v3/domains/c63acf42aaad40ceb6ce9125f0310c78 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:25.986573 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e05528-0f73-49e2-8b36-b09b493a3031 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:26.010682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:26.084630 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e05528-0f73-49e2-8b36-b09b493a3031 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:26.116533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b02e59-e302-482b-9e35-4f44b324f500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:26.117120 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b02e59-e302-482b-9e35-4f44b324f500 None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:26.117401 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b02e59-e302-482b-9e35-4f44b324f500 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:26.117674 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b02e59-e302-482b-9e35-4f44b324f500 None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:26.118324 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b02e59-e302-482b-9e35-4f44b324f500 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:26.119732 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b02e59-e302-482b-9e35-4f44b324f500 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:26.145915 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 601/1173] 10.222.0.81 () {60 vars in 1141 bytes} [Mon Sep 19 11:12:25 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:12:28.281209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916dc893-77e7-4e39-97ae-8e02f2bf3301 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:28.281752 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916dc893-77e7-4e39-97ae-8e02f2bf3301 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:28.281752 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916dc893-77e7-4e39-97ae-8e02f2bf3301 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:28.283080 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 573/1174] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:12:28.288290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b7d02-2e35-41ae-a731-97fa9e6e28cb None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:28.288733 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b7d02-2e35-41ae-a731-97fa9e6e28cb None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:28.288829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b7d02-2e35-41ae-a731-97fa9e6e28cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:28.306469 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-6c5b7d02-2e35-41ae-a731-97fa9e6e28cb None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:12:28.307645 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-6c5b7d02-2e35-41ae-a731-97fa9e6e28cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:12:28.371709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 602/1175] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:12:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:12:28.648001 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916dc893-77e7-4e39-97ae-8e02f2bf3301 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:28.742269 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:28.749628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916dc893-77e7-4e39-97ae-8e02f2bf3301 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:28.782758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4341a68-891c-4ff7-b027-ac1e7a93ae3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:28.783373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4341a68-891c-4ff7-b027-ac1e7a93ae3e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:28.783655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4341a68-891c-4ff7-b027-ac1e7a93ae3e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:28.783962 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4341a68-891c-4ff7-b027-ac1e7a93ae3e None admin] PATH_INFO: `/v3/domains/c63acf42aaad40ceb6ce9125f0310c78` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:28.785115 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4341a68-891c-4ff7-b027-ac1e7a93ae3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=c63acf42aaad40ceb6ce9125f0310c78)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:28.786553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4341a68-891c-4ff7-b027-ac1e7a93ae3e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:28.788163 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 574/1176] 10.222.0.81 () {56 vars in 1183 bytes} [Mon Sep 19 11:12:28 2022] GET /identity/v3/domains/c63acf42aaad40ceb6ce9125f0310c78 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:28.793921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b7d02-2e35-41ae-a731-97fa9e6e28cb None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:28.802322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:28.890491 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b7d02-2e35-41ae-a731-97fa9e6e28cb None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:28.922635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e67a56-c148-4ece-b650-b9e3756a8c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:28.923161 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e67a56-c148-4ece-b650-b9e3756a8c82 None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:28.923435 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e67a56-c148-4ece-b650-b9e3756a8c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:28.923699 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e67a56-c148-4ece-b650-b9e3756a8c82 None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:28.924364 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e67a56-c148-4ece-b650-b9e3756a8c82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:28.925819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e67a56-c148-4ece-b650-b9e3756a8c82 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:28.947947 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 603/1177] 10.222.0.81 () {60 vars in 1132 bytes} [Mon Sep 19 11:12:28 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:12:31.136318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466995e8-1ebe-4fd1-a59d-2e3fab40b010 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:31.136843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466995e8-1ebe-4fd1-a59d-2e3fab40b010 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:31.136843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466995e8-1ebe-4fd1-a59d-2e3fab40b010 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:31.138210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 575/1178] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:12:31.143567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d6c3a-2fbd-4fd0-bb0a-6e21b1ad5fce None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:31.144008 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d6c3a-2fbd-4fd0-bb0a-6e21b1ad5fce None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:31.144140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d6c3a-2fbd-4fd0-bb0a-6e21b1ad5fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:31.162352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-2e6d6c3a-2fbd-4fd0-bb0a-6e21b1ad5fce None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:12:31.163534 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2e6d6c3a-2fbd-4fd0-bb0a-6e21b1ad5fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:12:31.217463 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:31.229923 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 604/1179] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:12:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:12:31.502777 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466995e8-1ebe-4fd1-a59d-2e3fab40b010 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:31.601681 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466995e8-1ebe-4fd1-a59d-2e3fab40b010 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:31.635008 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d721b-1543-4afa-b092-e1f8dbdd97ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:31.635623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d721b-1543-4afa-b092-e1f8dbdd97ff None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:31.635931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d721b-1543-4afa-b092-e1f8dbdd97ff None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:31.636207 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d721b-1543-4afa-b092-e1f8dbdd97ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:31.645941 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d721b-1543-4afa-b092-e1f8dbdd97ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:12:31.650755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-109d721b-1543-4afa-b092-e1f8dbdd97ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:12:31.650755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:12:31.650755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:12:31.650755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:12:31.650755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:12:31.650755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:12:31.650755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:12:31.650755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:12:31.650755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:12:31.650755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:12:31.650755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:12:31.650755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:12:31.650755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:12:31.650755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:12:31.650755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:12:31.650755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:12:31.651731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:12:31.651731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:12:31.651731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:12:31.651731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:12:31.651731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:12:31.651731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:12:31.651731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:12:31.651731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:12:31.651731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:12:31.651731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:12:31.651731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:12:31.651731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:12:31.651731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:12:31.651731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:12:31.651731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:12:31.651731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:12:31.652928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:12:31.652928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:12:31.652928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:12:31.652928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:12:31.652928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:12:31.652928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:12:31.652928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:12:31.652928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:12:31.652928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:12:31.652928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:12:31.652928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:12:31.652928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:12:31.652928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 576/1180] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:12:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:12:31.657921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d6c3a-2fbd-4fd0-bb0a-6e21b1ad5fce None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:31.756889 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d6c3a-2fbd-4fd0-bb0a-6e21b1ad5fce None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:31.789868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb03ff-9e4f-4c8b-8320-4afe890c81c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:31.790464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb03ff-9e4f-4c8b-8320-4afe890c81c5 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:31.790744 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb03ff-9e4f-4c8b-8320-4afe890c81c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:31.791017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb03ff-9e4f-4c8b-8320-4afe890c81c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:31.791702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb03ff-9e4f-4c8b-8320-4afe890c81c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:12:31.792031 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb03ff-9e4f-4c8b-8320-4afe890c81c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:31.793390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb03ff-9e4f-4c8b-8320-4afe890c81c5 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:31.801674 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 605/1181] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:12:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:31.807560 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d721b-1543-4afa-b092-e1f8dbdd97ff None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:31.833480 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:31.905150 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d721b-1543-4afa-b092-e1f8dbdd97ff None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:31.938114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d492a5-000f-4d24-80d3-6cb898702b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:31.938720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d492a5-000f-4d24-80d3-6cb898702b90 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:31.938996 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d492a5-000f-4d24-80d3-6cb898702b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:31.939264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d492a5-000f-4d24-80d3-6cb898702b90 None admin] PATH_INFO: `/v3/users/7a5094735fc84e9784dedb18f71d016d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:31.951974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d492a5-000f-4d24-80d3-6cb898702b90 None admin] RBAC: Authorizing `identity:get_user(user_id=7a5094735fc84e9784dedb18f71d016d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:31.953404 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d492a5-000f-4d24-80d3-6cb898702b90 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:31.954992 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 577/1182] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:12:31 2022] GET /identity/v3/users/7a5094735fc84e9784dedb18f71d016d => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:31.960672 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb03ff-9e4f-4c8b-8320-4afe890c81c5 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:32.014005 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:32.057013 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb03ff-9e4f-4c8b-8320-4afe890c81c5 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:32.089907 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411ac0c-7688-49ad-89f0-34b24020ebfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:32.090491 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411ac0c-7688-49ad-89f0-34b24020ebfe None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:32.090768 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411ac0c-7688-49ad-89f0-34b24020ebfe None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:32.091039 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411ac0c-7688-49ad-89f0-34b24020ebfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:32.154687 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411ac0c-7688-49ad-89f0-34b24020ebfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:32.156136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411ac0c-7688-49ad-89f0-34b24020ebfe None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:32.207491 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:32.251443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 606/1183] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:12:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:12:32.257309 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d492a5-000f-4d24-80d3-6cb898702b90 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:32.336471 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:32.354168 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d492a5-000f-4d24-80d3-6cb898702b90 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:32.387940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcb1ab-06cb-44b9-a6d1-9d04219253cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:32.388734 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcb1ab-06cb-44b9-a6d1-9d04219253cc None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:32.389017 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcb1ab-06cb-44b9-a6d1-9d04219253cc None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:32.389289 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcb1ab-06cb-44b9-a6d1-9d04219253cc None admin] PATH_INFO: `/v3/projects/30a2341d1d934b929df0b76da8a205a3` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:32.390429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfcb1ab-06cb-44b9-a6d1-9d04219253cc None admin] RBAC: Authorizing `identity:get_project(project_id=30a2341d1d934b929df0b76da8a205a3)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:32.391889 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfcb1ab-06cb-44b9-a6d1-9d04219253cc None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:32.393544 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 578/1184] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:12:32 2022] GET /identity/v3/projects/30a2341d1d934b929df0b76da8a205a3 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:32.400222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411ac0c-7688-49ad-89f0-34b24020ebfe None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:32.495373 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411ac0c-7688-49ad-89f0-34b24020ebfe None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:32.528009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f25df9-46b7-41f5-84b6-8290fec35e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:32.528595 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f25df9-46b7-41f5-84b6-8290fec35e5a None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:32.528873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f25df9-46b7-41f5-84b6-8290fec35e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:32.529145 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f25df9-46b7-41f5-84b6-8290fec35e5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:32.529854 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f25df9-46b7-41f5-84b6-8290fec35e5a None admin] RBAC: Adding query filter params (role.id=753f8b35d5f0458d8f62d0ad6e28033f, scope.project.id=30a2341d1d934b929df0b76da8a205a3, user.id=7a5094735fc84e9784dedb18f71d016d) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:12:32.530161 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f25df9-46b7-41f5-84b6-8290fec35e5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:32.531561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f25df9-46b7-41f5-84b6-8290fec35e5a None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:32.536908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 607/1185] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:12:32 2022] GET /identity/v3/role_assignments?user.id=7a5094735fc84e9784dedb18f71d016d&scope.project.id=30a2341d1d934b929df0b76da8a205a3&role.id=753f8b35d5f0458d8f62d0ad6e28033f => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:34.627416 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ff0a6-ea4d-43a9-b040-fc349873b8a6 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:34.628020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ff0a6-ea4d-43a9-b040-fc349873b8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:34.628020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ff0a6-ea4d-43a9-b040-fc349873b8a6 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:34.629351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 579/1186] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:12:34.635088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d496462-5e69-4931-a51d-0923942bc969 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:34.635469 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d496462-5e69-4931-a51d-0923942bc969 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:34.635635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d496462-5e69-4931-a51d-0923942bc969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:34.653555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-3d496462-5e69-4931-a51d-0923942bc969 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:12:34.654748 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3d496462-5e69-4931-a51d-0923942bc969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:12:34.667735 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:34.720592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 608/1187] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:12:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:12:34.989751 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ff0a6-ea4d-43a9-b040-fc349873b8a6 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:35.088402 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ff0a6-ea4d-43a9-b040-fc349873b8a6 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:35.120337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f62ae-cd06-47d5-8118-98ce65ff78e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:35.120944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f62ae-cd06-47d5-8118-98ce65ff78e3 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:35.121218 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f62ae-cd06-47d5-8118-98ce65ff78e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:35.121485 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f62ae-cd06-47d5-8118-98ce65ff78e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:35.131599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f62ae-cd06-47d5-8118-98ce65ff78e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:12:35.136292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-120f62ae-cd06-47d5-8118-98ce65ff78e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:12:35.136292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:12:35.136292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:12:35.136292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:12:35.136292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:12:35.136292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:12:35.136292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:12:35.136292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:12:35.136292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:12:35.136292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:12:35.136292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:12:35.136292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:12:35.136292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:12:35.136292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:12:35.136292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:12:35.136292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:12:35.138274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:12:35.138274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:12:35.138274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:12:35.138274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:12:35.138274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:12:35.138274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:12:35.138274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:12:35.138274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:12:35.138274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:12:35.138274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:12:35.138274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:12:35.138274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:12:35.138274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:12:35.138274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:12:35.138274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:12:35.138274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:12:35.139136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:12:35.139136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:12:35.139136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:12:35.139136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:12:35.139136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:12:35.139136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:12:35.139136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:12:35.139136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:12:35.139136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:12:35.139136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:12:35.139136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:12:35.139136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:12:35.139136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 580/1188] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:12:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:12:35.143387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d496462-5e69-4931-a51d-0923942bc969 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:35.240017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d496462-5e69-4931-a51d-0923942bc969 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:35.273478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d7117-b163-4c69-bdb9-54ad903784eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:35.274074 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d7117-b163-4c69-bdb9-54ad903784eb None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:35.274354 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d7117-b163-4c69-bdb9-54ad903784eb None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:35.274629 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d7117-b163-4c69-bdb9-54ad903784eb None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:35.275288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d7117-b163-4c69-bdb9-54ad903784eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:12:35.275593 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d7117-b163-4c69-bdb9-54ad903784eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:35.277003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d7117-b163-4c69-bdb9-54ad903784eb None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:35.284949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 609/1189] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:12:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:35.290547 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f62ae-cd06-47d5-8118-98ce65ff78e3 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:35.303472 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:35.388517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f62ae-cd06-47d5-8118-98ce65ff78e3 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:35.421148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2eb977-69b3-429e-a723-eea29ffaf4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:35.421754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2eb977-69b3-429e-a723-eea29ffaf4cc None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:35.422033 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2eb977-69b3-429e-a723-eea29ffaf4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:35.422300 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2eb977-69b3-429e-a723-eea29ffaf4cc None admin] PATH_INFO: `/v3/users/7a5094735fc84e9784dedb18f71d016d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:35.423406 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2eb977-69b3-429e-a723-eea29ffaf4cc None admin] RBAC: Authorizing `identity:get_user(user_id=7a5094735fc84e9784dedb18f71d016d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:35.424854 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2eb977-69b3-429e-a723-eea29ffaf4cc None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:35.426931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 581/1190] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:12:35 2022] GET /identity/v3/users/7a5094735fc84e9784dedb18f71d016d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:35.433657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d7117-b163-4c69-bdb9-54ad903784eb None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:35.450160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:35.537336 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d7117-b163-4c69-bdb9-54ad903784eb None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:35.570122 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9fc77-0269-4ec8-93d2-94b019fb313f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:35.570739 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9fc77-0269-4ec8-93d2-94b019fb313f None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:35.571019 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9fc77-0269-4ec8-93d2-94b019fb313f None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:35.571290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9fc77-0269-4ec8-93d2-94b019fb313f None admin] PATH_INFO: `/v3/projects/30a2341d1d934b929df0b76da8a205a3` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:35.572483 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9fc77-0269-4ec8-93d2-94b019fb313f None admin] RBAC: Authorizing `identity:get_project(project_id=30a2341d1d934b929df0b76da8a205a3)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:35.573948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9fc77-0269-4ec8-93d2-94b019fb313f None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:35.575583 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 610/1191] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:12:35 2022] GET /identity/v3/projects/30a2341d1d934b929df0b76da8a205a3 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:12:35.581976 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2eb977-69b3-429e-a723-eea29ffaf4cc None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:35.656878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:35.679579 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2eb977-69b3-429e-a723-eea29ffaf4cc None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:35.712286 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b10438-3184-4617-9c1d-1c59baca9cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:35.712926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b10438-3184-4617-9c1d-1c59baca9cad None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:35.713201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b10438-3184-4617-9c1d-1c59baca9cad None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:35.713472 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b10438-3184-4617-9c1d-1c59baca9cad None admin] PATH_INFO: `/v3/projects/30a2341d1d934b929df0b76da8a205a3/users/7a5094735fc84e9784dedb18f71d016d/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:35.715327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b10438-3184-4617-9c1d-1c59baca9cad None admin] RBAC: Authorizing `identity:create_grant(project_id=30a2341d1d934b929df0b76da8a205a3, user_id=7a5094735fc84e9784dedb18f71d016d, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:35.716771 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b10438-3184-4617-9c1d-1c59baca9cad None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:35.729499 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 582/1192] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:12:35 2022] PUT /identity/v3/projects/30a2341d1d934b929df0b76da8a205a3/users/7a5094735fc84e9784dedb18f71d016d/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:12:37.841408 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f6f88-83bb-425c-9261-2cf6e70d287f None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:37.841935 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f6f88-83bb-425c-9261-2cf6e70d287f None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:37.841935 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f6f88-83bb-425c-9261-2cf6e70d287f None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:37.843270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 611/1193] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:12:37.849126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d226f75-ab1f-4822-be07-fa0627e403fb None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:37.849547 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d226f75-ab1f-4822-be07-fa0627e403fb None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:37.849658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d226f75-ab1f-4822-be07-fa0627e403fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:37.867578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-7d226f75-ab1f-4822-be07-fa0627e403fb None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:12:37.868783 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7d226f75-ab1f-4822-be07-fa0627e403fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:12:37.932212 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 583/1194] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:12:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:12:38.218672 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f6f88-83bb-425c-9261-2cf6e70d287f None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:38.297315 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:38.319503 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f6f88-83bb-425c-9261-2cf6e70d287f None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:38.352581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243089b8-5107-4cb7-b2b2-2336daedcefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:38.353191 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243089b8-5107-4cb7-b2b2-2336daedcefd None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:38.353470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243089b8-5107-4cb7-b2b2-2336daedcefd None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:38.353745 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243089b8-5107-4cb7-b2b2-2336daedcefd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:38.364515 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243089b8-5107-4cb7-b2b2-2336daedcefd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:12:38.369283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-243089b8-5107-4cb7-b2b2-2336daedcefd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:12:38.369283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:12:38.369283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:12:38.369283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:12:38.369283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:12:38.369283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:12:38.369283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:12:38.369283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:12:38.369283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:12:38.369283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:12:38.369283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:12:38.369283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:12:38.369283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:12:38.369283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:12:38.369283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:12:38.369283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:12:38.370417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:12:38.370417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:12:38.370417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:12:38.370417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:12:38.370417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:12:38.370417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:12:38.370417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:12:38.370417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:12:38.370417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:12:38.370417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:12:38.370417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:12:38.370417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:12:38.370417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:12:38.370417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:12:38.370417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:12:38.370417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:12:38.371295 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:12:38.371295 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:12:38.371295 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:12:38.371295 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:12:38.371295 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:12:38.371295 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:12:38.371295 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:12:38.371295 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:12:38.371295 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:12:38.371295 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:12:38.371295 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:12:38.371295 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:12:38.371295 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 612/1195] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:12:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:12:38.376338 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d226f75-ab1f-4822-be07-fa0627e403fb None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:38.440255 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:38.473640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d226f75-ab1f-4822-be07-fa0627e403fb None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:38.506858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf1555-bbee-42b9-8d49-55d5e08824bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:38.507453 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf1555-bbee-42b9-8d49-55d5e08824bd None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:38.507735 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf1555-bbee-42b9-8d49-55d5e08824bd None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:38.508040 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf1555-bbee-42b9-8d49-55d5e08824bd None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:38.508699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf1555-bbee-42b9-8d49-55d5e08824bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:12:38.509005 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf1555-bbee-42b9-8d49-55d5e08824bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:38.510376 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf1555-bbee-42b9-8d49-55d5e08824bd None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:38.518104 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 584/1196] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:12:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:38.523954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243089b8-5107-4cb7-b2b2-2336daedcefd None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:38.619595 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243089b8-5107-4cb7-b2b2-2336daedcefd None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:38.651885 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f3139-bc24-41b0-b4fd-5cdb0a112fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:38.652503 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f3139-bc24-41b0-b4fd-5cdb0a112fea None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:38.652784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f3139-bc24-41b0-b4fd-5cdb0a112fea None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:38.653060 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f3139-bc24-41b0-b4fd-5cdb0a112fea None admin] PATH_INFO: `/v3/users/7a5094735fc84e9784dedb18f71d016d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:38.654191 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f3139-bc24-41b0-b4fd-5cdb0a112fea None admin] RBAC: Authorizing `identity:get_user(user_id=7a5094735fc84e9784dedb18f71d016d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:38.655612 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f3139-bc24-41b0-b4fd-5cdb0a112fea None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:38.657209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 613/1197] 10.222.0.81 () {56 vars in 1177 bytes} [Mon Sep 19 11:12:38 2022] GET /identity/v3/users/7a5094735fc84e9784dedb18f71d016d => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:38.663265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf1555-bbee-42b9-8d49-55d5e08824bd None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:38.762619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf1555-bbee-42b9-8d49-55d5e08824bd None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:38.790127 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:38.796118 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5a38b-6bf7-414d-9f68-e7040b5dbe12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:38.796706 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5a38b-6bf7-414d-9f68-e7040b5dbe12 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:38.796983 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5a38b-6bf7-414d-9f68-e7040b5dbe12 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:38.797248 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5a38b-6bf7-414d-9f68-e7040b5dbe12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:38.861021 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5a38b-6bf7-414d-9f68-e7040b5dbe12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:38.862978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5a38b-6bf7-414d-9f68-e7040b5dbe12 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:38.960248 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 585/1198] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:12:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:12:38.966287 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f3139-bc24-41b0-b4fd-5cdb0a112fea None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:38.976199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:39.066511 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f3139-bc24-41b0-b4fd-5cdb0a112fea None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:39.099982 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ba830-1cf8-4787-b280-733384f2e6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:39.100594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ba830-1cf8-4787-b280-733384f2e6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:39.100869 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ba830-1cf8-4787-b280-733384f2e6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:39.101135 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ba830-1cf8-4787-b280-733384f2e6c3 None admin] PATH_INFO: `/v3/projects/30a2341d1d934b929df0b76da8a205a3` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:39.102223 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03ba830-1cf8-4787-b280-733384f2e6c3 None admin] RBAC: Authorizing `identity:get_project(project_id=30a2341d1d934b929df0b76da8a205a3)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:39.103636 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03ba830-1cf8-4787-b280-733384f2e6c3 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:39.105335 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 614/1199] 10.222.0.81 () {56 vars in 1186 bytes} [Mon Sep 19 11:12:38 2022] GET /identity/v3/projects/30a2341d1d934b929df0b76da8a205a3 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:39.111490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5a38b-6bf7-414d-9f68-e7040b5dbe12 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:12:39.140064 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:39.210233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5a38b-6bf7-414d-9f68-e7040b5dbe12 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:12:39.243429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e255b-0f25-4edf-b6b9-6b79e82c98ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:12:39.244056 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e255b-0f25-4edf-b6b9-6b79e82c98ce None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:12:39.244335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e255b-0f25-4edf-b6b9-6b79e82c98ce None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:12:39.244605 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e255b-0f25-4edf-b6b9-6b79e82c98ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:12:39.245317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5e255b-0f25-4edf-b6b9-6b79e82c98ce None admin] RBAC: Adding query filter params (role.id=753f8b35d5f0458d8f62d0ad6e28033f, scope.project.id=30a2341d1d934b929df0b76da8a205a3, user.id=7a5094735fc84e9784dedb18f71d016d) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:12:39.245624 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5e255b-0f25-4edf-b6b9-6b79e82c98ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:12:39.247038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5e255b-0f25-4edf-b6b9-6b79e82c98ce None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:12:39.252637 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 586/1200] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:12:39 2022] GET /identity/v3/role_assignments?user.id=7a5094735fc84e9784dedb18f71d016d&scope.project.id=30a2341d1d934b929df0b76da8a205a3&role.id=753f8b35d5f0458d8f62d0ad6e28033f => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:12:49.095370 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=63 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:12:49.250107 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=53 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:34.714659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb85a5-b9a6-49c5-a5ab-badcd579b7d3 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:34.715271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb85a5-b9a6-49c5-a5ab-badcd579b7d3 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:34.715271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb85a5-b9a6-49c5-a5ab-badcd579b7d3 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:34.716786 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 615/1201] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:13:34.723077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8685adc-ac3b-4e5a-97fc-093c7d33b166 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:34.723591 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8685adc-ac3b-4e5a-97fc-093c7d33b166 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:34.723700 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8685adc-ac3b-4e5a-97fc-093c7d33b166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:34.748464 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-b8685adc-ac3b-4e5a-97fc-093c7d33b166 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:13:34.748464 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b8685adc-ac3b-4e5a-97fc-093c7d33b166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:13:34.839738 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 587/1202] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:13:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:13:35.117823 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb85a5-b9a6-49c5-a5ab-badcd579b7d3 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:35.266189 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb85a5-b9a6-49c5-a5ab-badcd579b7d3 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:35.307156 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf21cc-671e-4c4d-a8c3-7b81597d7e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:35.307828 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf21cc-671e-4c4d-a8c3-7b81597d7e8a None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:35.308088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf21cc-671e-4c4d-a8c3-7b81597d7e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:35.308360 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf21cc-671e-4c4d-a8c3-7b81597d7e8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:35.324192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf21cc-671e-4c4d-a8c3-7b81597d7e8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:35.325638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf21cc-671e-4c4d-a8c3-7b81597d7e8a None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:35.347244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-7faf21cc-671e-4c4d-a8c3-7b81597d7e8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:35.347244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:35.347244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:35.347244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:35.347244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:35.347244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:35.347244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:35.347244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:35.347244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:35.347244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:35.347244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:35.347244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:35.347244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:13:35.347244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:13:35.347244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:13:35.347244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:13:35.348736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:35.348736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:35.348736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:35.348736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:35.348736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:35.348736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:35.348736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:35.348736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:35.348736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:35.348736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:35.348736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:35.348736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:35.348736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:35.348736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:35.348736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:35.348736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:35.348736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:13:35.349729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:13:35.349729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:35.349729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:13:35.349729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 616/1203] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:13:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:35.357189 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8685adc-ac3b-4e5a-97fc-093c7d33b166 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:35.499474 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8685adc-ac3b-4e5a-97fc-093c7d33b166 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:35.537937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:35.546315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9efd6-2c02-4a7e-9c6f-fb6b83b9e1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:35.547056 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9efd6-2c02-4a7e-9c6f-fb6b83b9e1b4 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:35.547486 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9efd6-2c02-4a7e-9c6f-fb6b83b9e1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:35.547976 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9efd6-2c02-4a7e-9c6f-fb6b83b9e1b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:35.548728 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e9efd6-2c02-4a7e-9c6f-fb6b83b9e1b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:35.549065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e9efd6-2c02-4a7e-9c6f-fb6b83b9e1b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:35.551077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e9efd6-2c02-4a7e-9c6f-fb6b83b9e1b4 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:35.567867 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 588/1204] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:35.570909 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf21cc-671e-4c4d-a8c3-7b81597d7e8a None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:35.643561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:35.721680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf21cc-671e-4c4d-a8c3-7b81597d7e8a None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:35.757098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9053f-c5f3-4b44-ab1c-beda81be8e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:35.757628 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9053f-c5f3-4b44-ab1c-beda81be8e9d None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:35.757905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9053f-c5f3-4b44-ab1c-beda81be8e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:35.758173 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9053f-c5f3-4b44-ab1c-beda81be8e9d None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:35.758841 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e9053f-c5f3-4b44-ab1c-beda81be8e9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:35.760256 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e9053f-c5f3-4b44-ab1c-beda81be8e9d None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:35.799793 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 617/1205] 10.222.0.81 () {60 vars in 1132 bytes} [Mon Sep 19 11:13:35 2022] POST /identity/v3/users => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:13:38.157871 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207ffa1-f773-4ffd-832b-4a60c596ef9a None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:38.158412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207ffa1-f773-4ffd-832b-4a60c596ef9a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:38.158412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207ffa1-f773-4ffd-832b-4a60c596ef9a None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:38.159742 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 589/1206] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:13:38.165633 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783706cc-cd95-4075-ad3a-f627036e8656 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:38.166105 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783706cc-cd95-4075-ad3a-f627036e8656 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:38.166314 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783706cc-cd95-4075-ad3a-f627036e8656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:38.190682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-783706cc-cd95-4075-ad3a-f627036e8656 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:13:38.192173 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-783706cc-cd95-4075-ad3a-f627036e8656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:13:38.274747 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 618/1207] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:13:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:13:38.556012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207ffa1-f773-4ffd-832b-4a60c596ef9a None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:38.676732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207ffa1-f773-4ffd-832b-4a60c596ef9a None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:38.720546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494aaeaf-9b67-42f9-9d66-503f1e5a8762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:38.721183 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494aaeaf-9b67-42f9-9d66-503f1e5a8762 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:38.721459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494aaeaf-9b67-42f9-9d66-503f1e5a8762 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:38.721755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494aaeaf-9b67-42f9-9d66-503f1e5a8762 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:38.734749 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494aaeaf-9b67-42f9-9d66-503f1e5a8762 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:13:38.740702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-494aaeaf-9b67-42f9-9d66-503f1e5a8762 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:13:38.740702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:38.740702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:38.740702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:38.740702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:38.740702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:38.740702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:38.740702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:38.740702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:38.740702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:38.740702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:38.740702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:38.740702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:13:38.740702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:13:38.740702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:13:38.740702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:13:38.741885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:13:38.741885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:13:38.741885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:13:38.741885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:13:38.741885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:38.741885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:38.741885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:38.741885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:38.741885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:38.741885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:38.741885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:38.741885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:38.741885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:38.741885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:38.741885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:38.741885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:38.742769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:38.742769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:38.742769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:38.742769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:38.742769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:13:38.742769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:13:38.742769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:13:38.742769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:13:38.742769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:13:38.742769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:13:38.742769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:13:38.742769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:13:38.742769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 590/1208] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:13:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:38.748153 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783706cc-cd95-4075-ad3a-f627036e8656 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:38.796824 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:38.862531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783706cc-cd95-4075-ad3a-f627036e8656 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:38.904934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579fbd1-5fb4-47d3-ad2a-4e40c8613deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:38.905539 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579fbd1-5fb4-47d3-ad2a-4e40c8613deb None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:38.905820 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579fbd1-5fb4-47d3-ad2a-4e40c8613deb None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:38.906089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579fbd1-5fb4-47d3-ad2a-4e40c8613deb None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:38.906747 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579fbd1-5fb4-47d3-ad2a-4e40c8613deb None admin] RBAC: Adding query filter params (name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:38.907041 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579fbd1-5fb4-47d3-ad2a-4e40c8613deb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:38.908561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579fbd1-5fb4-47d3-ad2a-4e40c8613deb None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:38.918416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 619/1209] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:13:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:38.924529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494aaeaf-9b67-42f9-9d66-503f1e5a8762 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:38.982572 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:39.041124 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494aaeaf-9b67-42f9-9d66-503f1e5a8762 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:39.079383 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289145cf-c7b6-4bab-9677-74e9ed7119ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:39.080168 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289145cf-c7b6-4bab-9677-74e9ed7119ab None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:39.080397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289145cf-c7b6-4bab-9677-74e9ed7119ab None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:39.080687 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289145cf-c7b6-4bab-9677-74e9ed7119ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:39.094318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289145cf-c7b6-4bab-9677-74e9ed7119ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:39.096152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289145cf-c7b6-4bab-9677-74e9ed7119ab None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:39.112625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-289145cf-c7b6-4bab-9677-74e9ed7119ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:39.112625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:39.112625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:39.112625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:39.112625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:39.112625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:39.112625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:39.112625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:39.112625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:39.112625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:39.112625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:39.112625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:39.112625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:13:39.112625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:13:39.112625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:13:39.112625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:13:39.113616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:39.113616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:39.113616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:39.113616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:39.113616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:39.113616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:39.113616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:39.113616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:39.113616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:39.113616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:39.113616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:39.113616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:39.113616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:39.113616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:39.113616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:39.113616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:39.113616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:13:39.114542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:13:39.114542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:39.114542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:13:39.118746 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 591/1210] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:13:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:39.120184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579fbd1-5fb4-47d3-ad2a-4e40c8613deb None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:39.228005 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:39.230570 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579fbd1-5fb4-47d3-ad2a-4e40c8613deb None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:39.272043 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb7cf9-bada-4bf9-bf2c-eb03d48c436c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:39.272710 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb7cf9-bada-4bf9-bf2c-eb03d48c436c None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:39.273086 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb7cf9-bada-4bf9-bf2c-eb03d48c436c None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:39.273405 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb7cf9-bada-4bf9-bf2c-eb03d48c436c None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:39.275242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb7cf9-bada-4bf9-bf2c-eb03d48c436c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:39.275242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb7cf9-bada-4bf9-bf2c-eb03d48c436c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:39.276013 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb7cf9-bada-4bf9-bf2c-eb03d48c436c None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:39.292899 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 620/1211] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:39.298641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289145cf-c7b6-4bab-9677-74e9ed7119ab None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:39.420236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289145cf-c7b6-4bab-9677-74e9ed7119ab None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:39.422458 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:39.460870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35205458-7ff4-486e-83fe-66bdba490702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:39.461552 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35205458-7ff4-486e-83fe-66bdba490702 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:39.461834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35205458-7ff4-486e-83fe-66bdba490702 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:39.462112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35205458-7ff4-486e-83fe-66bdba490702 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:39.470696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35205458-7ff4-486e-83fe-66bdba490702 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:39.472199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35205458-7ff4-486e-83fe-66bdba490702 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:39.483820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-35205458-7ff4-486e-83fe-66bdba490702 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:13:39.483820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:39.483820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:39.483820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:39.483820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:39.483820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:39.483820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:39.483820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:39.483820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:39.483820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:39.483820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:39.483820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:39.483820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:13:39.483820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:13:39.483820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:13:39.483820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:13:39.484873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:39.484873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:39.484873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:13:39.484873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:13:39.484873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:13:39.484873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:13:39.484873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:39.484873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:39.484873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:39.484873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:39.484873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:39.484873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:39.484873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:39.484873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:39.484873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:39.484873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:39.484873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:39.484873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:39.485878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:39.485878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:39.485878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:13:39.485878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:13:39.485878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:13:39.485878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:13:39.485878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:13:39.485878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:13:39.485878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:13:39.485878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:13:39.485878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 592/1212] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:13:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:39.490924 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb7cf9-bada-4bf9-bf2c-eb03d48c436c None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:39.610148 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb7cf9-bada-4bf9-bf2c-eb03d48c436c None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:39.652209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abdc3c3-2a63-4fc8-b1cd-5b236d9dc969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:39.655235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdc3c3-2a63-4fc8-b1cd-5b236d9dc969 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:39.655235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdc3c3-2a63-4fc8-b1cd-5b236d9dc969 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:39.655235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdc3c3-2a63-4fc8-b1cd-5b236d9dc969 None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:39.655235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abdc3c3-2a63-4fc8-b1cd-5b236d9dc969 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:39.655235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abdc3c3-2a63-4fc8-b1cd-5b236d9dc969 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:39.655727 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abdc3c3-2a63-4fc8-b1cd-5b236d9dc969 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:39.675645 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 621/1213] 10.222.0.81 () {56 vars in 1143 bytes} [Mon Sep 19 11:13:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:13:39.681609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35205458-7ff4-486e-83fe-66bdba490702 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:39.816090 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35205458-7ff4-486e-83fe-66bdba490702 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:39.853063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a559df9-985f-43b9-98a7-5f8ab6b04bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:39.853734 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a559df9-985f-43b9-98a7-5f8ab6b04bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:39.853963 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a559df9-985f-43b9-98a7-5f8ab6b04bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:39.854274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a559df9-985f-43b9-98a7-5f8ab6b04bf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:39.892347 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:39.935696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a559df9-985f-43b9-98a7-5f8ab6b04bf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:39.937227 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a559df9-985f-43b9-98a7-5f8ab6b04bf2 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:40.045100 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 593/1214] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:13:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:13:40.051811 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abdc3c3-2a63-4fc8-b1cd-5b236d9dc969 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:40.073172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:40.162153 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abdc3c3-2a63-4fc8-b1cd-5b236d9dc969 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:40.197909 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9aa389-4a21-4dcf-9ddb-db494c65e55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:40.199118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aa389-4a21-4dcf-9ddb-db494c65e55d None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:40.199620 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aa389-4a21-4dcf-9ddb-db494c65e55d None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:40.200133 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aa389-4a21-4dcf-9ddb-db494c65e55d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:40.211018 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9aa389-4a21-4dcf-9ddb-db494c65e55d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:40.212740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9aa389-4a21-4dcf-9ddb-db494c65e55d None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:40.227293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-8c9aa389-4a21-4dcf-9ddb-db494c65e55d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:40.227293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:40.227293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:40.227293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:40.227293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:40.227293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:40.227293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:40.227293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:40.227293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:40.227293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:40.227293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:40.227293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:40.227293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:13:40.227293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:13:40.227293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:13:40.227293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:13:40.228757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:40.228757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:40.228757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:40.228757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:40.228757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:40.228757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:40.228757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:40.228757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:40.228757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:40.228757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:40.228757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:40.228757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:40.228757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:40.228757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:40.228757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:40.228757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:40.228757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:13:40.229710 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:13:40.229710 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:40.229710 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:13:40.229710 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 622/1215] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:13:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:40.236197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a559df9-985f-43b9-98a7-5f8ab6b04bf2 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:40.316813 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:40.370101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a559df9-985f-43b9-98a7-5f8ab6b04bf2 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:40.408549 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5362fd-abb0-4b37-8955-581a15110939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:40.409487 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5362fd-abb0-4b37-8955-581a15110939 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:40.409932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5362fd-abb0-4b37-8955-581a15110939 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:40.410209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5362fd-abb0-4b37-8955-581a15110939 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:40.411011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5362fd-abb0-4b37-8955-581a15110939 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:40.411279 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5362fd-abb0-4b37-8955-581a15110939 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:40.412734 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5362fd-abb0-4b37-8955-581a15110939 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:40.428469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 594/1216] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:40.434158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9aa389-4a21-4dcf-9ddb-db494c65e55d None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:40.523653 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:40.564841 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9aa389-4a21-4dcf-9ddb-db494c65e55d None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:40.607509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ec19c-b8ab-4748-bace-266fdbd13659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:40.608731 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ec19c-b8ab-4748-bace-266fdbd13659 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:40.609104 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ec19c-b8ab-4748-bace-266fdbd13659 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:40.609367 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ec19c-b8ab-4748-bace-266fdbd13659 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:40.617723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ec19c-b8ab-4748-bace-266fdbd13659 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:40.619262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ec19c-b8ab-4748-bace-266fdbd13659 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:40.632666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-8f2ec19c-b8ab-4748-bace-266fdbd13659 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:13:40.632666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:40.632666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:40.632666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:40.632666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:40.632666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:40.632666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:40.632666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:40.632666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:40.632666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:40.632666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:40.632666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:40.632666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:13:40.632666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:13:40.632666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:13:40.634006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:13:40.634006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:40.634006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:40.634006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:40.634006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:40.634006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:40.634006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:40.634006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:40.634006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:40.634006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:40.634006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:40.634006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:40.634006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:40.634006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:40.634006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:40.634006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:40.634006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:40.634950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:13:40.634950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:13:40.634950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:13:40.634950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:13:40.634950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:13:40.634950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:13:40.634950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:13:40.634950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:13:40.634950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 623/1217] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:13:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:13:40.640628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5362fd-abb0-4b37-8955-581a15110939 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:40.767173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5362fd-abb0-4b37-8955-581a15110939 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:40.792706 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:40.809529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af99ba-29ad-4329-a8e8-0bb1d89bd131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:40.810320 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af99ba-29ad-4329-a8e8-0bb1d89bd131 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:40.810597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af99ba-29ad-4329-a8e8-0bb1d89bd131 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:40.810882 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af99ba-29ad-4329-a8e8-0bb1d89bd131 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:40.811679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af99ba-29ad-4329-a8e8-0bb1d89bd131 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:40.812011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af99ba-29ad-4329-a8e8-0bb1d89bd131 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:40.813511 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af99ba-29ad-4329-a8e8-0bb1d89bd131 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:40.834805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 595/1218] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:13:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:40.836344 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ec19c-b8ab-4748-bace-266fdbd13659 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:40.964568 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ec19c-b8ab-4748-bace-266fdbd13659 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:40.995520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:41.002975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803996a-3ae7-43b0-bc57-cd29595b59ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:41.003675 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803996a-3ae7-43b0-bc57-cd29595b59ef None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:41.003979 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803996a-3ae7-43b0-bc57-cd29595b59ef None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:41.004250 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803996a-3ae7-43b0-bc57-cd29595b59ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:41.005083 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803996a-3ae7-43b0-bc57-cd29595b59ef None admin] RBAC: Adding query filter params (role.id=af9d39307d714527b14b2045ab466e7a, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=6e7a18dc4a4d4d35a05f1aafbaaf9c31) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:41.005424 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803996a-3ae7-43b0-bc57-cd29595b59ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:41.006997 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803996a-3ae7-43b0-bc57-cd29595b59ef None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:41.013130 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 624/1219] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:13:40 2022] GET /identity/v3/role_assignments?user.id=6e7a18dc4a4d4d35a05f1aafbaaf9c31&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=af9d39307d714527b14b2045ab466e7a => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:43.211676 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68825c71-75d9-4e3f-a26a-16a1555103fa None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:43.212286 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68825c71-75d9-4e3f-a26a-16a1555103fa None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:43.212404 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68825c71-75d9-4e3f-a26a-16a1555103fa None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:43.213952 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 596/1220] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:13:43.219488 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1943e50-b342-4080-b5e9-f9e764a62944 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:43.220111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1943e50-b342-4080-b5e9-f9e764a62944 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:43.220111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1943e50-b342-4080-b5e9-f9e764a62944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:43.238674 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-a1943e50-b342-4080-b5e9-f9e764a62944 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:13:43.239918 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a1943e50-b342-4080-b5e9-f9e764a62944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:13:43.314631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 625/1221] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:13:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:13:43.588099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68825c71-75d9-4e3f-a26a-16a1555103fa None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:43.702565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68825c71-75d9-4e3f-a26a-16a1555103fa None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:43.741087 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecc8d1-cbd6-45bf-9aec-9f0852f9f598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:43.741849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecc8d1-cbd6-45bf-9aec-9f0852f9f598 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:43.742094 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecc8d1-cbd6-45bf-9aec-9f0852f9f598 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:43.742425 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecc8d1-cbd6-45bf-9aec-9f0852f9f598 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:43.753576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecc8d1-cbd6-45bf-9aec-9f0852f9f598 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:13:43.758944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-02ecc8d1-cbd6-45bf-9aec-9f0852f9f598 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:13:43.758944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:43.758944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:43.758944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:43.758944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:43.758944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:43.758944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:43.758944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:43.758944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:43.758944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:43.758944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:43.758944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:43.758944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:13:43.758944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:13:43.758944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:13:43.758944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:13:43.760354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:13:43.760354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:13:43.760354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:13:43.760354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:13:43.760354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:43.760354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:43.760354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:43.760354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:43.760354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:43.760354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:43.760354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:43.760354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:43.760354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:43.760354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:43.760354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:43.760354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:43.761494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:43.761494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:43.761494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:43.761494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:43.761494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:13:43.761494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:13:43.761494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:13:43.761494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:13:43.761494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:13:43.761494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:13:43.761494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:13:43.761494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:13:43.761494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 597/1222] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:13:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:13:43.767177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1943e50-b342-4080-b5e9-f9e764a62944 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:43.875987 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1943e50-b342-4080-b5e9-f9e764a62944 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:43.901191 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:43.913689 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9a428-4fef-4883-baf4-48893e65e9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:43.914367 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9a428-4fef-4883-baf4-48893e65e9e1 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:43.914806 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9a428-4fef-4883-baf4-48893e65e9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:43.915113 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9a428-4fef-4883-baf4-48893e65e9e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:43.915844 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9a428-4fef-4883-baf4-48893e65e9e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:43.916145 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9a428-4fef-4883-baf4-48893e65e9e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:43.917663 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9a428-4fef-4883-baf4-48893e65e9e1 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:43.925496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 626/1223] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:13:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:43.931251 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecc8d1-cbd6-45bf-9aec-9f0852f9f598 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:43.978476 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:44.037456 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecc8d1-cbd6-45bf-9aec-9f0852f9f598 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:44.073978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad2360-b9a2-4be3-9590-b5081b0e44a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:44.074602 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad2360-b9a2-4be3-9590-b5081b0e44a1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:44.074897 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad2360-b9a2-4be3-9590-b5081b0e44a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:44.075152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad2360-b9a2-4be3-9590-b5081b0e44a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:44.085599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad2360-b9a2-4be3-9590-b5081b0e44a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:44.087289 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad2360-b9a2-4be3-9590-b5081b0e44a1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:44.102599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-a6ad2360-b9a2-4be3-9590-b5081b0e44a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:44.102599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:44.102599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:44.102599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:44.102599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:44.102599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:44.102599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:44.102599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:44.102599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:44.102599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:44.102599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:44.102599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:44.102599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:13:44.102599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:13:44.102599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:13:44.102599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:13:44.103836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:44.103836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:44.103836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:44.103836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:44.103836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:44.103836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:44.103836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:44.103836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:44.103836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:44.103836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:44.103836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:44.103836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:44.103836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:44.103836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:44.103836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:44.103836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:44.103836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:13:44.105044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:13:44.105044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:44.105044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:13:44.108342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 598/1224] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:13:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:44.110566 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9a428-4fef-4883-baf4-48893e65e9e1 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:44.225674 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9a428-4fef-4883-baf4-48893e65e9e1 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:44.260016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50235fb7-1b26-479d-8c5d-9257815d77e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:44.260623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50235fb7-1b26-479d-8c5d-9257815d77e5 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:44.260913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50235fb7-1b26-479d-8c5d-9257815d77e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:44.261167 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50235fb7-1b26-479d-8c5d-9257815d77e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:44.261837 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50235fb7-1b26-479d-8c5d-9257815d77e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:44.262136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50235fb7-1b26-479d-8c5d-9257815d77e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:44.265564 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50235fb7-1b26-479d-8c5d-9257815d77e5 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:44.278517 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 627/1225] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:44.285478 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad2360-b9a2-4be3-9590-b5081b0e44a1 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:44.400284 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:44.407401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad2360-b9a2-4be3-9590-b5081b0e44a1 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:44.444638 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d630368-11b9-40e5-b76e-68378f7c3802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:44.445269 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d630368-11b9-40e5-b76e-68378f7c3802 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:44.445548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d630368-11b9-40e5-b76e-68378f7c3802 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:44.445819 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d630368-11b9-40e5-b76e-68378f7c3802 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:44.453835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d630368-11b9-40e5-b76e-68378f7c3802 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:44.455272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d630368-11b9-40e5-b76e-68378f7c3802 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:44.466316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-9d630368-11b9-40e5-b76e-68378f7c3802 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:13:44.466316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:44.466316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:44.466316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:44.466316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:44.466316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:44.466316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:44.466316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:44.466316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:44.466316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:44.466316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:44.466316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:44.466316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:13:44.466316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:13:44.466316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:13:44.466316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:13:44.467890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:44.467890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:44.467890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:13:44.467890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:13:44.467890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:13:44.467890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:13:44.467890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:44.467890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:44.467890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:44.467890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:44.467890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:44.467890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:44.467890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:44.467890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:44.467890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:44.467890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:44.467890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:44.467890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:44.469201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:44.469201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:44.469201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:13:44.469201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:13:44.469201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:13:44.469201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:13:44.469201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:13:44.469201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:13:44.469201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:13:44.469201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:13:44.469201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 599/1226] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:13:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:13:44.473523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50235fb7-1b26-479d-8c5d-9257815d77e5 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:44.522764 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:44.583495 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50235fb7-1b26-479d-8c5d-9257815d77e5 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:44.617307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c06db-1bf5-4b22-a03a-cb9513677afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:44.617904 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c06db-1bf5-4b22-a03a-cb9513677afd None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:44.618182 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c06db-1bf5-4b22-a03a-cb9513677afd None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:44.618449 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c06db-1bf5-4b22-a03a-cb9513677afd None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:44.619166 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c06db-1bf5-4b22-a03a-cb9513677afd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:44.619468 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c06db-1bf5-4b22-a03a-cb9513677afd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:44.620914 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c06db-1bf5-4b22-a03a-cb9513677afd None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:44.637603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 628/1227] 10.222.0.81 () {56 vars in 1143 bytes} [Mon Sep 19 11:13:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:44.644354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d630368-11b9-40e5-b76e-68378f7c3802 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:44.756820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d630368-11b9-40e5-b76e-68378f7c3802 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:44.793038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fc6d5-a6ff-4b95-96df-bb52537ffc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:44.793677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fc6d5-a6ff-4b95-96df-bb52537ffc70 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:44.793961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fc6d5-a6ff-4b95-96df-bb52537ffc70 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:44.794330 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fc6d5-a6ff-4b95-96df-bb52537ffc70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:44.806686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0fc6d5-a6ff-4b95-96df-bb52537ffc70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:44.808241 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0fc6d5-a6ff-4b95-96df-bb52537ffc70 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:44.824932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-0d0fc6d5-a6ff-4b95-96df-bb52537ffc70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:44.824932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:44.824932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:44.824932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:44.824932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:44.824932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:44.824932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:44.824932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:44.824932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:44.824932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:44.824932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:44.824932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:44.824932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:13:44.824932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:13:44.824932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:13:44.824932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:13:44.825950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:44.825950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:44.825950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:44.825950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:44.825950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:44.825950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:44.825950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:44.825950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:44.825950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:44.825950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:44.825950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:44.825950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:44.825950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:44.825950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:44.825950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:44.825950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:44.825950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:13:44.827396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:13:44.827396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:44.827396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:13:44.827396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 600/1228] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:44.833072 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c06db-1bf5-4b22-a03a-cb9513677afd None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:44.940554 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c06db-1bf5-4b22-a03a-cb9513677afd None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:44.943899 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:44.978723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2bfdc-b40a-47fe-99d7-6d6c0d4ea46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:44.979374 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2bfdc-b40a-47fe-99d7-6d6c0d4ea46d None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:44.979664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2bfdc-b40a-47fe-99d7-6d6c0d4ea46d None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:44.980062 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2bfdc-b40a-47fe-99d7-6d6c0d4ea46d None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:44.980712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2bfdc-b40a-47fe-99d7-6d6c0d4ea46d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:44.981078 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2bfdc-b40a-47fe-99d7-6d6c0d4ea46d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:44.982725 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2bfdc-b40a-47fe-99d7-6d6c0d4ea46d None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:44.993875 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 629/1229] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:45.000251 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fc6d5-a6ff-4b95-96df-bb52537ffc70 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:45.010474 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:45.108034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fc6d5-a6ff-4b95-96df-bb52537ffc70 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:45.143996 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37843421-833d-486e-a046-65ec654c43be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:45.144762 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37843421-833d-486e-a046-65ec654c43be None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:45.145055 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37843421-833d-486e-a046-65ec654c43be None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:45.145468 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37843421-833d-486e-a046-65ec654c43be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:45.153419 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37843421-833d-486e-a046-65ec654c43be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:45.155315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37843421-833d-486e-a046-65ec654c43be None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:45.168869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-37843421-833d-486e-a046-65ec654c43be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:13:45.168869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:45.168869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:45.168869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:45.168869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:45.168869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:45.168869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:45.168869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:45.168869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:45.168869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:45.168869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:45.168869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:45.168869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:13:45.168869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:13:45.168869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:13:45.169956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:13:45.169956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:45.169956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:45.169956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:45.169956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:45.169956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:45.169956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:45.169956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:45.169956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:45.169956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:45.169956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:45.169956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:45.169956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:45.169956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:45.169956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:45.169956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:45.169956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:45.171423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:13:45.171423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:13:45.171423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:13:45.171423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:13:45.171423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:13:45.171423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:13:45.171423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:13:45.171423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:13:45.171423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 601/1230] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:13:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:45.176445 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2bfdc-b40a-47fe-99d7-6d6c0d4ea46d None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:45.287671 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2bfdc-b40a-47fe-99d7-6d6c0d4ea46d None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:45.334838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18f21b-2a3b-4e6c-ba30-900c4d84dbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:45.335528 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18f21b-2a3b-4e6c-ba30-900c4d84dbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:45.335743 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18f21b-2a3b-4e6c-ba30-900c4d84dbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:45.336049 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18f21b-2a3b-4e6c-ba30-900c4d84dbb1 None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:45.336757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18f21b-2a3b-4e6c-ba30-900c4d84dbb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:45.337059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18f21b-2a3b-4e6c-ba30-900c4d84dbb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:45.338476 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18f21b-2a3b-4e6c-ba30-900c4d84dbb1 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:45.351714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 630/1231] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:13:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:45.357960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37843421-833d-486e-a046-65ec654c43be None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:45.431062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:45.471444 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37843421-833d-486e-a046-65ec654c43be None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:45.509350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc3cc2-9bd7-4621-890c-fe51a38e583b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:45.510062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc3cc2-9bd7-4621-890c-fe51a38e583b None admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:45.510382 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc3cc2-9bd7-4621-890c-fe51a38e583b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:45.510676 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc3cc2-9bd7-4621-890c-fe51a38e583b None admin] PATH_INFO: `/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/6e7a18dc4a4d4d35a05f1aafbaaf9c31/roles/af9d39307d714527b14b2045ab466e7a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:45.525844 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc3cc2-9bd7-4621-890c-fe51a38e583b None admin] RBAC: Authorizing `identity:create_grant(project_id=ed9a0b626e3e4b93b391158e981ec586, user_id=6e7a18dc4a4d4d35a05f1aafbaaf9c31, role_id=af9d39307d714527b14b2045ab466e7a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:45.527285 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc3cc2-9bd7-4621-890c-fe51a38e583b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:45.539849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 602/1232] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:13:45 2022] PUT /identity/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/6e7a18dc4a4d4d35a05f1aafbaaf9c31/roles/af9d39307d714527b14b2045ab466e7a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:13:47.680089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b525b4-6fe1-4f5d-aa23-7e5fac97c9b4 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:47.680573 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b525b4-6fe1-4f5d-aa23-7e5fac97c9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:47.680776 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b525b4-6fe1-4f5d-aa23-7e5fac97c9b4 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:47.682054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 631/1233] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:13:47.687362 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b88d2-f292-4faa-87b7-b933677ecbb8 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:47.687931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b88d2-f292-4faa-87b7-b933677ecbb8 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:47.687931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b88d2-f292-4faa-87b7-b933677ecbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:47.708126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a44b88d2-f292-4faa-87b7-b933677ecbb8 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:13:47.709478 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a44b88d2-f292-4faa-87b7-b933677ecbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:13:47.783034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 603/1234] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:13:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:13:48.051518 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b525b4-6fe1-4f5d-aa23-7e5fac97c9b4 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:48.079917 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:48.161770 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b525b4-6fe1-4f5d-aa23-7e5fac97c9b4 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:48.199503 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535f7d9-afa5-4e65-82c8-6dec01a2aadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:48.200182 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535f7d9-afa5-4e65-82c8-6dec01a2aadb None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:48.200460 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535f7d9-afa5-4e65-82c8-6dec01a2aadb None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:48.200733 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535f7d9-afa5-4e65-82c8-6dec01a2aadb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:48.211741 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535f7d9-afa5-4e65-82c8-6dec01a2aadb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:13:48.216576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-1535f7d9-afa5-4e65-82c8-6dec01a2aadb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:13:48.216576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:48.216576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:48.216576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:48.216576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:48.216576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:48.216576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:48.216576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:48.216576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:48.216576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:48.216576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:48.216576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:48.216576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:13:48.216576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:13:48.216576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:13:48.216576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:13:48.217896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:13:48.217896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:13:48.217896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:13:48.217896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:13:48.217896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:48.217896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:48.217896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:48.217896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:48.217896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:48.217896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:48.217896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:48.217896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:48.217896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:48.217896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:48.217896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:48.217896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:48.219523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:48.219523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:48.219523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:48.219523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:48.219523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:13:48.219523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:13:48.219523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:13:48.219523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:13:48.219523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:13:48.219523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:13:48.219523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 11:13:48.219523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:13:48.219523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 632/1235] 10.222.0.81 () {56 vars in 1102 bytes} [Mon Sep 19 11:13:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:48.225370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b88d2-f292-4faa-87b7-b933677ecbb8 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:48.279239 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:48.358106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b88d2-f292-4faa-87b7-b933677ecbb8 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:48.395753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ea1f8-cf2f-4feb-85e0-a3a584240e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:48.396387 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ea1f8-cf2f-4feb-85e0-a3a584240e03 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:48.396699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ea1f8-cf2f-4feb-85e0-a3a584240e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:48.396983 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ea1f8-cf2f-4feb-85e0-a3a584240e03 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:48.397667 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ea1f8-cf2f-4feb-85e0-a3a584240e03 None admin] RBAC: Adding query filter params (name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:48.397971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ea1f8-cf2f-4feb-85e0-a3a584240e03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:48.399435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ea1f8-cf2f-4feb-85e0-a3a584240e03 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:48.409353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 604/1236] 10.222.0.81 () {56 vars in 1103 bytes} [Mon Sep 19 11:13:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:48.415560 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535f7d9-afa5-4e65-82c8-6dec01a2aadb None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:48.516199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:48.534083 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535f7d9-afa5-4e65-82c8-6dec01a2aadb None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:48.572681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a11f7d-4793-41a3-879a-c82980fb466d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:48.573590 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a11f7d-4793-41a3-879a-c82980fb466d None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:48.573883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a11f7d-4793-41a3-879a-c82980fb466d None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:48.574121 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a11f7d-4793-41a3-879a-c82980fb466d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:48.585805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a11f7d-4793-41a3-879a-c82980fb466d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:48.587252 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a11f7d-4793-41a3-879a-c82980fb466d None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:48.602529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-69a11f7d-4793-41a3-879a-c82980fb466d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:48.602529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:48.602529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:48.602529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:48.602529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:48.602529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:48.602529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:48.602529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:48.602529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:48.602529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:48.602529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:48.602529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:48.602529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:13:48.602529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:13:48.602529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:13:48.602529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:13:48.603953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:48.603953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:48.603953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:48.603953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:48.603953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:48.603953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:48.603953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:48.603953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:48.603953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:48.603953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:48.603953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:48.603953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:48.603953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:48.603953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:48.603953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:48.603953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:48.603953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:13:48.605171 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:13:48.605171 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:48.605171 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:13:48.605171 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 633/1237] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:13:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:48.609957 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ea1f8-cf2f-4feb-85e0-a3a584240e03 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:48.718893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ea1f8-cf2f-4feb-85e0-a3a584240e03 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:48.728427 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:48.756472 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84837833-45d8-4587-9d3b-412d22ea6dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:48.757280 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84837833-45d8-4587-9d3b-412d22ea6dbd None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:48.757596 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84837833-45d8-4587-9d3b-412d22ea6dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:48.758000 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84837833-45d8-4587-9d3b-412d22ea6dbd None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:48.758762 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84837833-45d8-4587-9d3b-412d22ea6dbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:48.759071 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84837833-45d8-4587-9d3b-412d22ea6dbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:48.760660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84837833-45d8-4587-9d3b-412d22ea6dbd None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:48.773869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 605/1238] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:48.780028 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a11f7d-4793-41a3-879a-c82980fb466d None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:48.896185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a11f7d-4793-41a3-879a-c82980fb466d None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:48.943823 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957493a5-9dec-4894-9ea0-9a3cc6332428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:48.944447 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957493a5-9dec-4894-9ea0-9a3cc6332428 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:48.944726 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957493a5-9dec-4894-9ea0-9a3cc6332428 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:48.944993 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957493a5-9dec-4894-9ea0-9a3cc6332428 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:48.947888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:48.954254 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957493a5-9dec-4894-9ea0-9a3cc6332428 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:48.955702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957493a5-9dec-4894-9ea0-9a3cc6332428 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:48.970387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-957493a5-9dec-4894-9ea0-9a3cc6332428 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:13:48.970387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:48.970387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:48.970387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:48.970387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:48.970387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:48.970387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:48.970387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:48.970387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:48.970387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:48.970387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:48.970387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:48.970387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:13:48.970387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:13:48.970387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:13:48.970387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:13:48.971635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:48.971635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:48.971635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:13:48.971635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:13:48.971635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:13:48.971635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:13:48.971635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:48.971635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:48.971635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:48.971635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:48.971635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:48.971635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:48.971635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:48.971635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:48.971635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:48.971635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:48.971635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:48.971635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:48.972751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:48.972751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:48.972751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:13:48.972751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:13:48.972751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:13:48.972751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:13:48.972751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:13:48.972751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:13:48.972751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:13:48.972751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:13:48.972751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 634/1239] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:13:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:13:48.980236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84837833-45d8-4587-9d3b-412d22ea6dbd None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:49.091975 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84837833-45d8-4587-9d3b-412d22ea6dbd None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:49.130900 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e9328-8a74-4b89-9405-d2ae908afdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:49.131782 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e9328-8a74-4b89-9405-d2ae908afdfe None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:49.132067 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e9328-8a74-4b89-9405-d2ae908afdfe None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:49.132358 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e9328-8a74-4b89-9405-d2ae908afdfe None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:49.133071 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e9328-8a74-4b89-9405-d2ae908afdfe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:49.133377 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e9328-8a74-4b89-9405-d2ae908afdfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:49.134813 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e9328-8a74-4b89-9405-d2ae908afdfe None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:49.152684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 606/1240] 10.222.0.81 () {56 vars in 1143 bytes} [Mon Sep 19 11:13:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:49.159252 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957493a5-9dec-4894-9ea0-9a3cc6332428 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:49.274318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957493a5-9dec-4894-9ea0-9a3cc6332428 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:49.315244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c5fcc-3c01-4c85-91d9-1f37f2197722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:49.315987 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c5fcc-3c01-4c85-91d9-1f37f2197722 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:49.316272 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c5fcc-3c01-4c85-91d9-1f37f2197722 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:49.316547 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c5fcc-3c01-4c85-91d9-1f37f2197722 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:49.391070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:49.400640 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c5fcc-3c01-4c85-91d9-1f37f2197722 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:49.402095 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c5fcc-3c01-4c85-91d9-1f37f2197722 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:49.509405 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 635/1241] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:13:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 11:13:49.516076 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e9328-8a74-4b89-9405-d2ae908afdfe None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:49.545485 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:49.625761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e9328-8a74-4b89-9405-d2ae908afdfe None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:49.666032 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348d98c-7636-4166-ba51-cfbdb1dcffff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:49.666696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348d98c-7636-4166-ba51-cfbdb1dcffff None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:49.666976 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348d98c-7636-4166-ba51-cfbdb1dcffff None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:49.667251 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348d98c-7636-4166-ba51-cfbdb1dcffff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:49.679457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348d98c-7636-4166-ba51-cfbdb1dcffff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:49.681096 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348d98c-7636-4166-ba51-cfbdb1dcffff None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:49.698537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-d348d98c-7636-4166-ba51-cfbdb1dcffff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:49.698537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:49.698537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:49.698537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:49.698537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:49.698537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:49.698537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:49.698537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:49.698537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:49.698537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:49.698537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:49.698537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:49.698537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:13:49.698537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:13:49.698537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:13:49.698537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:13:49.699797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:49.699797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:49.699797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:49.699797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:49.699797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:49.699797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:49.699797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:49.699797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:49.699797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:49.699797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:49.699797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:49.699797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:49.699797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:49.699797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:49.699797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:49.699797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:49.699797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:13:49.701482 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:13:49.701482 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:49.701482 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:13:49.701482 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 607/1242] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:49.706415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c5fcc-3c01-4c85-91d9-1f37f2197722 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:49.815105 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:49.821629 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c5fcc-3c01-4c85-91d9-1f37f2197722 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:49.860411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c11d0-7c49-493b-9170-d69170fef951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:49.861018 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c11d0-7c49-493b-9170-d69170fef951 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:49.861298 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c11d0-7c49-493b-9170-d69170fef951 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:49.861566 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c11d0-7c49-493b-9170-d69170fef951 None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:49.862202 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c11d0-7c49-493b-9170-d69170fef951 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:49.862497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c11d0-7c49-493b-9170-d69170fef951 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:49.863895 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c11d0-7c49-493b-9170-d69170fef951 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:49.876049 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 636/1243] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:49.882050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348d98c-7636-4166-ba51-cfbdb1dcffff None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:49.974046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:49.999977 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348d98c-7636-4166-ba51-cfbdb1dcffff None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:50.037496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc474f-8680-4b6b-b8cb-22e3d0015b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:50.038127 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc474f-8680-4b6b-b8cb-22e3d0015b45 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:50.038392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc474f-8680-4b6b-b8cb-22e3d0015b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:50.038657 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc474f-8680-4b6b-b8cb-22e3d0015b45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:50.045536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc474f-8680-4b6b-b8cb-22e3d0015b45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:50.047030 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc474f-8680-4b6b-b8cb-22e3d0015b45 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:50.059839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-5bdc474f-8680-4b6b-b8cb-22e3d0015b45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:13:50.059839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:50.059839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:50.059839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:50.059839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:50.059839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:50.059839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:50.059839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:50.059839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:50.059839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:50.059839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:50.059839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:50.059839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:13:50.059839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:13:50.059839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:13:50.061565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:13:50.061565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:50.061565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:50.061565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:50.061565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:50.061565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:50.061565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:50.061565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:50.061565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:50.061565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:50.061565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:50.061565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:50.061565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:50.061565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:50.061565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:50.061565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:50.061565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:50.062768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:13:50.062768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:13:50.062768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:13:50.062768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:13:50.062768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:13:50.062768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:13:50.062768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:13:50.062768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:13:50.062768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 608/1244] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:13:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:50.070060 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c11d0-7c49-493b-9170-d69170fef951 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:50.185165 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c11d0-7c49-493b-9170-d69170fef951 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:50.223163 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d877e-fcae-4ab2-b854-b76be5c9e7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:50.223756 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d877e-fcae-4ab2-b854-b76be5c9e7be None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:50.224081 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d877e-fcae-4ab2-b854-b76be5c9e7be None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:50.224401 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d877e-fcae-4ab2-b854-b76be5c9e7be None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:50.225102 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d877e-fcae-4ab2-b854-b76be5c9e7be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:50.225450 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d877e-fcae-4ab2-b854-b76be5c9e7be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:50.226943 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d877e-fcae-4ab2-b854-b76be5c9e7be None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:50.239753 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 637/1245] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:13:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:13:50.245925 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc474f-8680-4b6b-b8cb-22e3d0015b45 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:50.368477 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc474f-8680-4b6b-b8cb-22e3d0015b45 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:50.410398 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0206f-bcef-438f-b8f2-1e93c554ce34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:50.411218 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0206f-bcef-438f-b8f2-1e93c554ce34 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:50.411502 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0206f-bcef-438f-b8f2-1e93c554ce34 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:50.411795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0206f-bcef-438f-b8f2-1e93c554ce34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:50.412594 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0206f-bcef-438f-b8f2-1e93c554ce34 None admin] RBAC: Adding query filter params (role.id=af9d39307d714527b14b2045ab466e7a, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=6e7a18dc4a4d4d35a05f1aafbaaf9c31) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:50.412888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0206f-bcef-438f-b8f2-1e93c554ce34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:50.415014 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0206f-bcef-438f-b8f2-1e93c554ce34 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:50.423043 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 609/1246] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:13:50 2022] GET /identity/v3/role_assignments?user.id=6e7a18dc4a4d4d35a05f1aafbaaf9c31&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=af9d39307d714527b14b2045ab466e7a => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:13:52.945833 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e83bb-6542-436f-ad08-7a568cd9dff1 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:52.946346 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e83bb-6542-436f-ad08-7a568cd9dff1 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:52.946346 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e83bb-6542-436f-ad08-7a568cd9dff1 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:52.947698 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 638/1247] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:13:52.953944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45532db8-83aa-4499-831c-92b3f1d3f5cb None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:52.954402 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45532db8-83aa-4499-831c-92b3f1d3f5cb None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:52.954514 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45532db8-83aa-4499-831c-92b3f1d3f5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:52.960926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:52.975064 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-45532db8-83aa-4499-831c-92b3f1d3f5cb None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:13:52.976339 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-45532db8-83aa-4499-831c-92b3f1d3f5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:13:53.052801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 610/1248] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:13:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:13:53.330707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e83bb-6542-436f-ad08-7a568cd9dff1 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:53.352812 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:53.445992 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e83bb-6542-436f-ad08-7a568cd9dff1 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:53.485287 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0faac-10c4-4164-b605-757875b77b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:53.485882 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0faac-10c4-4164-b605-757875b77b73 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:53.486169 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0faac-10c4-4164-b605-757875b77b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:53.486444 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0faac-10c4-4164-b605-757875b77b73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:53.497248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b0faac-10c4-4164-b605-757875b77b73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:13:53.503009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-d0b0faac-10c4-4164-b605-757875b77b73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:13:53.503009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:53.503009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:53.503009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:53.503009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:53.503009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:53.503009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:53.503009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:53.503009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:53.503009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:53.503009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:53.503009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:53.503009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:13:53.503009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:13:53.503009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:13:53.503009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:13:53.504432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:13:53.504432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:13:53.504432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:13:53.504432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:13:53.504432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:53.504432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:53.504432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:53.504432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:53.504432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:53.504432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:53.504432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:53.504432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:53.504432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:53.504432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:53.504432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:53.504432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:53.505714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:53.505714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:53.505714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:53.505714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:53.505714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:13:53.505714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:13:53.505714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:13:53.505714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:13:53.505714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:13:53.505714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:13:53.505714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:13:53.505714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:13:53.505714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 639/1249] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:13:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:53.510357 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45532db8-83aa-4499-831c-92b3f1d3f5cb None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:53.616631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45532db8-83aa-4499-831c-92b3f1d3f5cb None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:53.649248 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:53.658585 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399c4ae-d007-4753-a125-18c8f199836f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:53.659154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399c4ae-d007-4753-a125-18c8f199836f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:53.659468 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399c4ae-d007-4753-a125-18c8f199836f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:53.659808 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399c4ae-d007-4753-a125-18c8f199836f None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:53.660926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2399c4ae-d007-4753-a125-18c8f199836f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:53.661269 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2399c4ae-d007-4753-a125-18c8f199836f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:53.662826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2399c4ae-d007-4753-a125-18c8f199836f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:53.673299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 611/1250] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:13:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:53.680102 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0faac-10c4-4164-b605-757875b77b73 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:53.779162 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:53.806057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0faac-10c4-4164-b605-757875b77b73 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:53.845432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3374963-b4c4-4616-a4d3-bb0352b0255f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:53.846058 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3374963-b4c4-4616-a4d3-bb0352b0255f None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:53.846333 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3374963-b4c4-4616-a4d3-bb0352b0255f None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:53.846599 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3374963-b4c4-4616-a4d3-bb0352b0255f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:53.857224 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3374963-b4c4-4616-a4d3-bb0352b0255f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:53.859089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3374963-b4c4-4616-a4d3-bb0352b0255f None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:53.874826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-a3374963-b4c4-4616-a4d3-bb0352b0255f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:53.874826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:53.874826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:53.874826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:53.874826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:53.874826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:53.874826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:53.874826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:53.874826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:53.874826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:53.874826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:53.874826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:53.874826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:13:53.874826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:13:53.874826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:13:53.874826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:13:53.876068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:53.876068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:53.876068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:53.876068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:53.876068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:53.876068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:53.876068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:53.876068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:53.876068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:53.876068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:53.876068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:53.876068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:53.876068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:53.876068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:53.876068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:53.876068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:53.876068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:13:53.877320 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:13:53.877320 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:53.877320 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:13:53.877320 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 640/1251] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:53.881561 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399c4ae-d007-4753-a125-18c8f199836f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:53.993550 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399c4ae-d007-4753-a125-18c8f199836f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:54.031413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3118dfb-facb-494b-bfe4-20d125779955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:54.032120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3118dfb-facb-494b-bfe4-20d125779955 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:54.032342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3118dfb-facb-494b-bfe4-20d125779955 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:54.032711 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3118dfb-facb-494b-bfe4-20d125779955 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:54.033563 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3118dfb-facb-494b-bfe4-20d125779955 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:54.033939 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3118dfb-facb-494b-bfe4-20d125779955 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:54.035426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3118dfb-facb-494b-bfe4-20d125779955 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:54.050092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 612/1252] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:54.056038 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3374963-b4c4-4616-a4d3-bb0352b0255f None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:54.168950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3374963-b4c4-4616-a4d3-bb0352b0255f None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:54.196394 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:54.205589 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db18f7-e7cc-46b7-acc9-dfd74753d5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:54.206232 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db18f7-e7cc-46b7-acc9-dfd74753d5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:54.206508 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db18f7-e7cc-46b7-acc9-dfd74753d5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:54.206778 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db18f7-e7cc-46b7-acc9-dfd74753d5e6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:54.215299 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db18f7-e7cc-46b7-acc9-dfd74753d5e6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:54.217509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db18f7-e7cc-46b7-acc9-dfd74753d5e6 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:54.229941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-77db18f7-e7cc-46b7-acc9-dfd74753d5e6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:13:54.229941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:54.229941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:54.229941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:54.229941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:54.229941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:54.229941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:54.229941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:54.229941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:54.229941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:54.229941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:54.229941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:54.229941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:13:54.229941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:13:54.229941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:13:54.229941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:13:54.231157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:54.231157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:54.231157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:13:54.231157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:13:54.231157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:13:54.231157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:13:54.231157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:54.231157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:54.231157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:54.231157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:54.231157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:54.231157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:54.231157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:54.231157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:54.231157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:54.231157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:54.231157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:54.231157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:54.232231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:54.232231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:54.232231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:13:54.232231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:13:54.232231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:13:54.232231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:13:54.232231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:13:54.232231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:13:54.232231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:13:54.232231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:13:54.232231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 641/1253] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:13:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:54.237979 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3118dfb-facb-494b-bfe4-20d125779955 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:54.299943 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:54.354984 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3118dfb-facb-494b-bfe4-20d125779955 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:54.393629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c58da6-098a-4e49-9a29-02c95d8dd17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:54.394239 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c58da6-098a-4e49-9a29-02c95d8dd17c None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:54.394515 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c58da6-098a-4e49-9a29-02c95d8dd17c None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:54.394782 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c58da6-098a-4e49-9a29-02c95d8dd17c None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:54.395516 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c58da6-098a-4e49-9a29-02c95d8dd17c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:54.395849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c58da6-098a-4e49-9a29-02c95d8dd17c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:54.397295 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c58da6-098a-4e49-9a29-02c95d8dd17c None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:54.416497 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 613/1254] 10.222.0.81 () {56 vars in 1143 bytes} [Mon Sep 19 11:13:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:54.422574 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db18f7-e7cc-46b7-acc9-dfd74753d5e6 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:54.527266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db18f7-e7cc-46b7-acc9-dfd74753d5e6 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:54.560830 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc4deb-02ea-471d-ace7-b2f02b5c8e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:54.561589 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc4deb-02ea-471d-ace7-b2f02b5c8e89 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:54.562566 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc4deb-02ea-471d-ace7-b2f02b5c8e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:54.562566 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc4deb-02ea-471d-ace7-b2f02b5c8e89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:54.616613 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:54.629725 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc4deb-02ea-471d-ace7-b2f02b5c8e89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:54.631138 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc4deb-02ea-471d-ace7-b2f02b5c8e89 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:54.731728 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 642/1255] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:13:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 11:13:54.738782 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c58da6-098a-4e49-9a29-02c95d8dd17c None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:54.838230 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c58da6-098a-4e49-9a29-02c95d8dd17c None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:54.851446 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:54.871941 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067626e-31fb-491c-a9be-6d9425fff18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:54.872609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067626e-31fb-491c-a9be-6d9425fff18e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:54.872888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067626e-31fb-491c-a9be-6d9425fff18e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:54.873191 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067626e-31fb-491c-a9be-6d9425fff18e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:54.882971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067626e-31fb-491c-a9be-6d9425fff18e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:54.884433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067626e-31fb-491c-a9be-6d9425fff18e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:54.899002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-d067626e-31fb-491c-a9be-6d9425fff18e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:54.899002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:54.899002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:54.899002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:54.899002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:54.899002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:54.899002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:54.899002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:54.899002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:54.899002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:54.899002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:54.899002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:54.899002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:13:54.899002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:13:54.899002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:13:54.899002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:13:54.900159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:54.900159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:54.900159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:54.900159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:54.900159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:54.900159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:54.900159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:54.900159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:54.900159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:54.900159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:54.900159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:54.900159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:54.900159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:54.900159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:54.900159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:54.900159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:54.900159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:13:54.901451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:13:54.901451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:54.901451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:13:54.901451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 614/1256] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:13:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:54.906398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc4deb-02ea-471d-ace7-b2f02b5c8e89 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:54.990432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:55.002298 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc4deb-02ea-471d-ace7-b2f02b5c8e89 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:55.035595 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c175a1d-c7d3-4675-b42c-9435c765981f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:55.036215 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c175a1d-c7d3-4675-b42c-9435c765981f None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:55.036498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c175a1d-c7d3-4675-b42c-9435c765981f None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:55.036772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c175a1d-c7d3-4675-b42c-9435c765981f None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:55.037407 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c175a1d-c7d3-4675-b42c-9435c765981f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:55.037710 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c175a1d-c7d3-4675-b42c-9435c765981f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:55.039059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c175a1d-c7d3-4675-b42c-9435c765981f None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:55.049793 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 643/1257] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:55.055278 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067626e-31fb-491c-a9be-6d9425fff18e None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:55.155284 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067626e-31fb-491c-a9be-6d9425fff18e None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:55.188030 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e916ee1-80f3-4746-a3f6-39ea39bf3eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:55.188641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e916ee1-80f3-4746-a3f6-39ea39bf3eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:55.188923 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e916ee1-80f3-4746-a3f6-39ea39bf3eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:55.189196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e916ee1-80f3-4746-a3f6-39ea39bf3eb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:55.195891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e916ee1-80f3-4746-a3f6-39ea39bf3eb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:55.197337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e916ee1-80f3-4746-a3f6-39ea39bf3eb1 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:55.209256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-5e916ee1-80f3-4746-a3f6-39ea39bf3eb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:13:55.209256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:55.209256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:55.209256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:55.209256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:55.209256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:55.209256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:55.209256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:55.209256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:55.209256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:55.209256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:55.209256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:55.209256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:13:55.209256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:13:55.209256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:13:55.210317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:13:55.210317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:55.210317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:55.210317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:55.210317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:55.210317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:55.210317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:55.210317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:55.210317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:55.210317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:55.210317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:55.210317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:55.210317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:55.210317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:55.210317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:55.210317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:55.210317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:55.211264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:13:55.211264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:13:55.211264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:13:55.211264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:13:55.211264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:13:55.211264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:13:55.211264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:13:55.211264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:13:55.211264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 615/1258] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:13:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:55.216234 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c175a1d-c7d3-4675-b42c-9435c765981f None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:55.314518 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c175a1d-c7d3-4675-b42c-9435c765981f None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:55.349387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a54bbf-460f-4db5-8820-9b87dd877c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:55.349972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a54bbf-460f-4db5-8820-9b87dd877c58 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:55.350248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a54bbf-460f-4db5-8820-9b87dd877c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:55.350513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a54bbf-460f-4db5-8820-9b87dd877c58 None admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:55.351188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a54bbf-460f-4db5-8820-9b87dd877c58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:55.351491 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a54bbf-460f-4db5-8820-9b87dd877c58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:55.352921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a54bbf-460f-4db5-8820-9b87dd877c58 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:55.364335 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 644/1259] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:13:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:55.370443 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e916ee1-80f3-4746-a3f6-39ea39bf3eb1 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:55.396456 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:55.469375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e916ee1-80f3-4746-a3f6-39ea39bf3eb1 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:55.501813 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33165f2a-e3c3-459b-8916-2c1f2a53509e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:55.502404 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33165f2a-e3c3-459b-8916-2c1f2a53509e None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:55.502683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33165f2a-e3c3-459b-8916-2c1f2a53509e None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:55.502957 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33165f2a-e3c3-459b-8916-2c1f2a53509e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:55.503667 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33165f2a-e3c3-459b-8916-2c1f2a53509e None admin] RBAC: Adding query filter params (role.id=753f8b35d5f0458d8f62d0ad6e28033f, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=6e7a18dc4a4d4d35a05f1aafbaaf9c31) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:55.504008 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33165f2a-e3c3-459b-8916-2c1f2a53509e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:55.505436 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33165f2a-e3c3-459b-8916-2c1f2a53509e None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:55.510629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 616/1260] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:13:55 2022] GET /identity/v3/role_assignments?user.id=6e7a18dc4a4d4d35a05f1aafbaaf9c31&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=753f8b35d5f0458d8f62d0ad6e28033f => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:57.606011 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a13e9-7cc5-42aa-bca4-b595d23432ac None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:57.606542 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a13e9-7cc5-42aa-bca4-b595d23432ac None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:57.606542 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a13e9-7cc5-42aa-bca4-b595d23432ac None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:57.607960 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 645/1261] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:13:57.612982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c252d1-2668-43b8-a041-d21ca182414a None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:57.613416 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c252d1-2668-43b8-a041-d21ca182414a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:57.613537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c252d1-2668-43b8-a041-d21ca182414a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:57.632860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-b3c252d1-2668-43b8-a041-d21ca182414a None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:13:57.634040 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b3c252d1-2668-43b8-a041-d21ca182414a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:13:57.698987 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 617/1262] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:13:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:13:57.966534 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a13e9-7cc5-42aa-bca4-b595d23432ac None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:57.977316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:58.065642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a13e9-7cc5-42aa-bca4-b595d23432ac None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:58.098489 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f11f9a-3427-4430-ac43-b7ddebb6d9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:58.099092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f11f9a-3427-4430-ac43-b7ddebb6d9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:58.099361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f11f9a-3427-4430-ac43-b7ddebb6d9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:58.099625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f11f9a-3427-4430-ac43-b7ddebb6d9a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:58.109273 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f11f9a-3427-4430-ac43-b7ddebb6d9a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:13:58.113941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-34f11f9a-3427-4430-ac43-b7ddebb6d9a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:13:58.113941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:58.113941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:58.113941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:58.113941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:58.113941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:58.113941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:58.113941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:58.113941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:58.113941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:58.113941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:58.113941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:58.113941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:13:58.113941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:13:58.113941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:13:58.113941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:13:58.115024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:13:58.115024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:13:58.115024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:13:58.115024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:13:58.115024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:58.115024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:58.115024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:58.115024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:58.115024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:58.115024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:58.115024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:58.115024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:58.115024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:58.115024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:58.115024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:58.115024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:58.115929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:58.115929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:58.115929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:58.115929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:58.115929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:13:58.115929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:13:58.115929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:13:58.115929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:13:58.115929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:13:58.115929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:13:58.115929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:13:58.115929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:13:58.115929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 646/1263] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:13:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:58.121007 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c252d1-2668-43b8-a041-d21ca182414a None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:58.139201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:58.218097 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c252d1-2668-43b8-a041-d21ca182414a None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:58.250530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78ecd9-d826-496a-bc04-6af0e62f0408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:58.251128 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78ecd9-d826-496a-bc04-6af0e62f0408 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:58.251409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78ecd9-d826-496a-bc04-6af0e62f0408 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:58.251682 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78ecd9-d826-496a-bc04-6af0e62f0408 None admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:58.252366 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78ecd9-d826-496a-bc04-6af0e62f0408 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:58.252670 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78ecd9-d826-496a-bc04-6af0e62f0408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:58.254029 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78ecd9-d826-496a-bc04-6af0e62f0408 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:58.261909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 618/1264] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:13:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:58.268064 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f11f9a-3427-4430-ac43-b7ddebb6d9a9 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:58.342536 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:58.377349 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f11f9a-3427-4430-ac43-b7ddebb6d9a9 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:58.409702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7da586-939c-4afc-89f1-5962008010a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:58.410313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7da586-939c-4afc-89f1-5962008010a0 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:58.410585 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7da586-939c-4afc-89f1-5962008010a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:58.410850 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7da586-939c-4afc-89f1-5962008010a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:58.420903 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7da586-939c-4afc-89f1-5962008010a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:58.422317 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7da586-939c-4afc-89f1-5962008010a0 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:58.436266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-9a7da586-939c-4afc-89f1-5962008010a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:58.436266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:58.436266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:58.436266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:58.436266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:58.436266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:58.436266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:58.436266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:58.436266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:58.436266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:58.436266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:58.436266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:58.436266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:13:58.436266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:13:58.436266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:13:58.436266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:13:58.437409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:58.437409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:58.437409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:58.437409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:58.437409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:58.437409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:58.437409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:58.437409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:58.437409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:58.437409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:58.437409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:58.437409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:58.437409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:58.437409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:58.437409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:58.437409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:58.437409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:13:58.438723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:13:58.438723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:58.438723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:13:58.438723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 647/1265] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:13:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 11:13:58.443107 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78ecd9-d826-496a-bc04-6af0e62f0408 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:58.529861 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:58.550349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78ecd9-d826-496a-bc04-6af0e62f0408 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:58.583742 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8e8c5-5765-4c05-8916-8d2184d0b8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:58.584354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8e8c5-5765-4c05-8916-8d2184d0b8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:58.584629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8e8c5-5765-4c05-8916-8d2184d0b8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:58.584894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8e8c5-5765-4c05-8916-8d2184d0b8e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:58.585523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8e8c5-5765-4c05-8916-8d2184d0b8e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:58.585820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8e8c5-5765-4c05-8916-8d2184d0b8e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:58.587172 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8e8c5-5765-4c05-8916-8d2184d0b8e0 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:58.598954 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 619/1266] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:58.604691 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7da586-939c-4afc-89f1-5962008010a0 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:58.702519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7da586-939c-4afc-89f1-5962008010a0 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:58.721833 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:58.735714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa372f-cd9c-42f3-8a58-7292554b2f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:58.736358 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa372f-cd9c-42f3-8a58-7292554b2f07 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:58.736639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa372f-cd9c-42f3-8a58-7292554b2f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:58.736927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa372f-cd9c-42f3-8a58-7292554b2f07 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:58.744067 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa372f-cd9c-42f3-8a58-7292554b2f07 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:58.745477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa372f-cd9c-42f3-8a58-7292554b2f07 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:58.756553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-26aa372f-cd9c-42f3-8a58-7292554b2f07 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:13:58.756553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:58.756553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:58.756553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:58.756553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:58.756553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:58.756553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:58.756553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:58.756553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:58.756553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:58.756553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:58.756553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:58.756553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:13:58.756553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:13:58.756553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:13:58.756553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:13:58.757594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:58.757594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:58.757594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:13:58.757594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:13:58.757594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:13:58.757594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:13:58.757594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:58.757594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:58.757594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:58.757594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:58.757594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:58.757594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:58.757594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:58.757594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:58.757594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:58.757594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:58.757594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:58.757594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:58.758681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:58.758681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:58.758681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:13:58.758681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:13:58.758681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:13:58.758681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:13:58.758681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:13:58.758681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:13:58.758681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:13:58.758681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:13:58.758681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 648/1267] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:13:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:58.764065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8e8c5-5765-4c05-8916-8d2184d0b8e0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:58.860776 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8e8c5-5765-4c05-8916-8d2184d0b8e0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:58.893685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcb001-f4bc-41d5-ba94-4bd0bcbe31e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:58.894276 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcb001-f4bc-41d5-ba94-4bd0bcbe31e0 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:58.894556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcb001-f4bc-41d5-ba94-4bd0bcbe31e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:58.894826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcb001-f4bc-41d5-ba94-4bd0bcbe31e0 None admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:58.895546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dcb001-f4bc-41d5-ba94-4bd0bcbe31e0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:58.895886 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dcb001-f4bc-41d5-ba94-4bd0bcbe31e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:58.897303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dcb001-f4bc-41d5-ba94-4bd0bcbe31e0 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:58.906166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:58.914488 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 620/1268] 10.222.0.81 () {56 vars in 1143 bytes} [Mon Sep 19 11:13:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:58.920280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa372f-cd9c-42f3-8a58-7292554b2f07 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:59.018296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa372f-cd9c-42f3-8a58-7292554b2f07 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:59.050309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c613fc0f-88b5-4fa4-92d5-712fc2c0988d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:59.050914 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613fc0f-88b5-4fa4-92d5-712fc2c0988d None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:59.051195 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613fc0f-88b5-4fa4-92d5-712fc2c0988d None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:59.051470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613fc0f-88b5-4fa4-92d5-712fc2c0988d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:59.062162 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c613fc0f-88b5-4fa4-92d5-712fc2c0988d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:59.063582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c613fc0f-88b5-4fa4-92d5-712fc2c0988d None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:59.077694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-c613fc0f-88b5-4fa4-92d5-712fc2c0988d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:59.077694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:59.077694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:59.077694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:59.077694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:59.077694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:59.077694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:59.077694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:59.077694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:59.077694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:59.077694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:59.077694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:59.077694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:13:59.077694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:13:59.077694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:13:59.077694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:13:59.078668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:59.078668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:59.078668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:59.078668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:59.078668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:59.078668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:59.078668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:59.078668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:59.078668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:59.078668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:59.078668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:59.078668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:59.078668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:59.078668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:59.078668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:59.078668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:59.078668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:13:59.079663 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:13:59.079663 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:13:59.079663 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:13:59.079663 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 649/1269] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:13:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:59.084666 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcb001-f4bc-41d5-ba94-4bd0bcbe31e0 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:59.181430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcb001-f4bc-41d5-ba94-4bd0bcbe31e0 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:59.214102 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29231351-d64b-41b3-ae7f-9331d97f81df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:59.214688 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29231351-d64b-41b3-ae7f-9331d97f81df None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:59.214967 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29231351-d64b-41b3-ae7f-9331d97f81df None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:59.215238 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29231351-d64b-41b3-ae7f-9331d97f81df None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:59.216419 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29231351-d64b-41b3-ae7f-9331d97f81df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:59.216723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29231351-d64b-41b3-ae7f-9331d97f81df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:59.218091 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29231351-d64b-41b3-ae7f-9331d97f81df None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:59.229035 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 621/1270] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:59.234574 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c613fc0f-88b5-4fa4-92d5-712fc2c0988d None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:59.266749 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:59.336815 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c613fc0f-88b5-4fa4-92d5-712fc2c0988d None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:59.373374 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e040a9a-5a35-4924-aa82-c618f520783e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:59.373978 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e040a9a-5a35-4924-aa82-c618f520783e None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:59.374247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e040a9a-5a35-4924-aa82-c618f520783e None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:59.374511 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e040a9a-5a35-4924-aa82-c618f520783e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:59.380940 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e040a9a-5a35-4924-aa82-c618f520783e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:59.382352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e040a9a-5a35-4924-aa82-c618f520783e None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:59.393144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-2e040a9a-5a35-4924-aa82-c618f520783e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:13:59.393144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:13:59.393144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:13:59.393144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:13:59.393144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:13:59.393144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:13:59.393144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:13:59.393144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:13:59.393144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:13:59.393144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:13:59.393144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:13:59.393144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:13:59.393144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:13:59.393144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:13:59.393144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:13:59.394184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:13:59.394184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:13:59.394184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:13:59.394184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:13:59.394184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:13:59.394184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:13:59.394184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:13:59.394184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:13:59.394184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:13:59.394184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:13:59.394184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:13:59.394184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:13:59.394184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:13:59.394184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:13:59.394184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:13:59.394184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:13:59.394184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:13:59.395467 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:13:59.395467 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:13:59.395467 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:13:59.395467 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:13:59.395467 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:13:59.395467 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:13:59.395467 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:13:59.395467 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:13:59.395467 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 650/1271] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:13:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:13:59.400078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29231351-d64b-41b3-ae7f-9331d97f81df None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:59.457641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:59.498788 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29231351-d64b-41b3-ae7f-9331d97f81df None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:59.531659 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6447aaa5-334d-4cf6-89f2-850957e758cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:59.532280 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447aaa5-334d-4cf6-89f2-850957e758cc None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:59.532560 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447aaa5-334d-4cf6-89f2-850957e758cc None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:59.532832 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6447aaa5-334d-4cf6-89f2-850957e758cc None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:59.533513 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6447aaa5-334d-4cf6-89f2-850957e758cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:13:59.533819 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6447aaa5-334d-4cf6-89f2-850957e758cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:59.535236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6447aaa5-334d-4cf6-89f2-850957e758cc None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:59.546830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 622/1272] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:13:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:13:59.552772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e040a9a-5a35-4924-aa82-c618f520783e None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:13:59.640675 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:13:59.649792 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e040a9a-5a35-4924-aa82-c618f520783e None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:13:59.684101 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c19401-6844-4ba0-a075-8f277c21ad9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:13:59.685278 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c19401-6844-4ba0-a075-8f277c21ad9b None admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:13:59.685557 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c19401-6844-4ba0-a075-8f277c21ad9b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:13:59.685828 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c19401-6844-4ba0-a075-8f277c21ad9b None admin] PATH_INFO: `/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/6e7a18dc4a4d4d35a05f1aafbaaf9c31/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:13:59.687753 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c19401-6844-4ba0-a075-8f277c21ad9b None admin] RBAC: Authorizing `identity:create_grant(project_id=ed9a0b626e3e4b93b391158e981ec586, user_id=6e7a18dc4a4d4d35a05f1aafbaaf9c31, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:13:59.689193 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c19401-6844-4ba0-a075-8f277c21ad9b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:13:59.702270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 651/1273] 10.222.0.81 () {58 vars in 1439 bytes} [Mon Sep 19 11:13:59 2022] PUT /identity/v3/projects/ed9a0b626e3e4b93b391158e981ec586/users/6e7a18dc4a4d4d35a05f1aafbaaf9c31/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:14:01.870074 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a9e2a-abc3-4299-8d18-a9b5add9e0c6 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:01.870689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a9e2a-abc3-4299-8d18-a9b5add9e0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:01.870689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a9e2a-abc3-4299-8d18-a9b5add9e0c6 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:01.872020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 623/1274] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:14:01.877633 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23da50d-65dd-40f8-8b0f-27b4a3ecb989 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:01.878056 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23da50d-65dd-40f8-8b0f-27b4a3ecb989 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:01.878166 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23da50d-65dd-40f8-8b0f-27b4a3ecb989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:01.897818 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-a23da50d-65dd-40f8-8b0f-27b4a3ecb989 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:14:01.899036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a23da50d-65dd-40f8-8b0f-27b4a3ecb989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:14:01.971286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 652/1275] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:14:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:14:02.244157 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a9e2a-abc3-4299-8d18-a9b5add9e0c6 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:02.358453 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a9e2a-abc3-4299-8d18-a9b5add9e0c6 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:02.376044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:14:02.394178 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359a3ed-a1eb-4594-8c97-72cc42834f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:02.394798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359a3ed-a1eb-4594-8c97-72cc42834f38 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:02.395074 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359a3ed-a1eb-4594-8c97-72cc42834f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:02.395343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359a3ed-a1eb-4594-8c97-72cc42834f38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:02.406141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c359a3ed-a1eb-4594-8c97-72cc42834f38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 11:14:02.410869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-c359a3ed-a1eb-4594-8c97-72cc42834f38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:14:02.410869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:14:02.410869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:14:02.410869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:14:02.410869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:14:02.410869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:14:02.410869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:14:02.410869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:14:02.410869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:14:02.410869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:14:02.410869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:14:02.410869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:14:02.410869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 11:14:02.410869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 11:14:02.410869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 11:14:02.410869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 11:14:02.411999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 11:14:02.411999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 11:14:02.411999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 11:14:02.411999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 11:14:02.411999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:14:02.411999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:14:02.411999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:14:02.411999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:14:02.411999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:14:02.411999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:14:02.411999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:14:02.411999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:14:02.411999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:14:02.411999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:14:02.411999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:14:02.411999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:14:02.413240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:14:02.413240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:14:02.413240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:14:02.413240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:14:02.413240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 11:14:02.413240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 11:14:02.413240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 11:14:02.413240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 11:14:02.413240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 11:14:02.413240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 11:14:02.413240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 11:14:02.413240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:14:02.413240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 624/1276] 10.222.0.81 () {56 vars in 1096 bytes} [Mon Sep 19 11:14:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:14:02.418423 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23da50d-65dd-40f8-8b0f-27b4a3ecb989 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:02.500051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:14:02.523504 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23da50d-65dd-40f8-8b0f-27b4a3ecb989 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:02.556348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da8128-5ee5-405a-b253-6b7fd8ce52aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:02.556932 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da8128-5ee5-405a-b253-6b7fd8ce52aa None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:02.557204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da8128-5ee5-405a-b253-6b7fd8ce52aa None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:02.557471 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da8128-5ee5-405a-b253-6b7fd8ce52aa None admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:02.558119 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da8128-5ee5-405a-b253-6b7fd8ce52aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:14:02.558416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da8128-5ee5-405a-b253-6b7fd8ce52aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:02.559854 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da8128-5ee5-405a-b253-6b7fd8ce52aa None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:02.567449 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 653/1277] 10.222.0.81 () {56 vars in 1099 bytes} [Mon Sep 19 11:14:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:14:02.573709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359a3ed-a1eb-4594-8c97-72cc42834f38 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:02.677157 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359a3ed-a1eb-4594-8c97-72cc42834f38 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:02.711333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c3321-8ded-486e-aa56-d19ee2c3213f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:02.711978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c3321-8ded-486e-aa56-d19ee2c3213f None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:02.712260 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c3321-8ded-486e-aa56-d19ee2c3213f None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:02.712532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c3321-8ded-486e-aa56-d19ee2c3213f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:02.722302 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c3321-8ded-486e-aa56-d19ee2c3213f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:02.723743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c3321-8ded-486e-aa56-d19ee2c3213f None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:02.737932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-ba6c3321-8ded-486e-aa56-d19ee2c3213f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:14:02.737932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:14:02.737932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:14:02.737932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:14:02.737932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:14:02.737932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:14:02.737932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:14:02.737932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:14:02.737932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:14:02.737932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:14:02.737932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:14:02.737932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:14:02.737932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:14:02.737932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:14:02.737932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:14:02.737932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:14:02.743169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:14:02.743169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:14:02.743169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:14:02.743169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:14:02.743169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:14:02.743169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:14:02.743169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:14:02.743169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:14:02.743169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:14:02.743169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:14:02.743169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:14:02.743169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:14:02.743169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:14:02.743169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:14:02.743169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:14:02.743169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:14:02.743169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:14:02.744245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:14:02.744245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:14:02.744245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:14:02.744245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 625/1278] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:14:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:14:02.744839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da8128-5ee5-405a-b253-6b7fd8ce52aa None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:02.842835 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da8128-5ee5-405a-b253-6b7fd8ce52aa None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:02.875772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d670bf-b2bd-4048-9abc-6dbc0e82f37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:02.876372 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d670bf-b2bd-4048-9abc-6dbc0e82f37d None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:02.876652 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d670bf-b2bd-4048-9abc-6dbc0e82f37d None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:02.876930 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d670bf-b2bd-4048-9abc-6dbc0e82f37d None admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:02.877559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d670bf-b2bd-4048-9abc-6dbc0e82f37d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:14:02.877863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d670bf-b2bd-4048-9abc-6dbc0e82f37d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:02.879217 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d670bf-b2bd-4048-9abc-6dbc0e82f37d None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:02.891083 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 654/1279] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:14:02.896722 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c3321-8ded-486e-aa56-d19ee2c3213f None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:02.926596 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:14:02.994222 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c3321-8ded-486e-aa56-d19ee2c3213f None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:03.027970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65232f03-2a7b-4cc3-b6b5-d93b434eaa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:03.028603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65232f03-2a7b-4cc3-b6b5-d93b434eaa16 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:03.028882 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65232f03-2a7b-4cc3-b6b5-d93b434eaa16 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:03.029155 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65232f03-2a7b-4cc3-b6b5-d93b434eaa16 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:03.036396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65232f03-2a7b-4cc3-b6b5-d93b434eaa16 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:03.037827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65232f03-2a7b-4cc3-b6b5-d93b434eaa16 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:03.049292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-65232f03-2a7b-4cc3-b6b5-d93b434eaa16 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:14:03.049292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:14:03.049292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:14:03.049292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:14:03.049292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:14:03.049292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:14:03.049292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:14:03.049292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:14:03.049292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:14:03.049292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:14:03.049292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:14:03.049292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:14:03.049292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 11:14:03.049292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 11:14:03.049292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 11:14:03.049292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 11:14:03.050375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:14:03.050375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:14:03.050375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 11:14:03.050375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:14:03.050375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 11:14:03.050375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 11:14:03.050375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:14:03.050375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:14:03.050375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:14:03.050375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:14:03.050375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:14:03.050375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:14:03.050375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:14:03.050375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:14:03.050375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:14:03.050375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:14:03.050375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:14:03.050375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:14:03.051709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:14:03.051709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:14:03.051709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 11:14:03.051709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 11:14:03.051709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 11:14:03.051709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 11:14:03.051709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 11:14:03.051709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 11:14:03.051709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 11:14:03.051709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:14:03.051709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 626/1280] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:14:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:14:03.056209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d670bf-b2bd-4048-9abc-6dbc0e82f37d None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:03.060514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:14:03.153140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d670bf-b2bd-4048-9abc-6dbc0e82f37d None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:03.186163 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68508ca1-c872-431d-97d7-7b1c9e262b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:03.186752 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68508ca1-c872-431d-97d7-7b1c9e262b9e None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:03.187036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68508ca1-c872-431d-97d7-7b1c9e262b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:03.187307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68508ca1-c872-431d-97d7-7b1c9e262b9e None admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:03.188048 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68508ca1-c872-431d-97d7-7b1c9e262b9e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:14:03.188352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68508ca1-c872-431d-97d7-7b1c9e262b9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:03.189753 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68508ca1-c872-431d-97d7-7b1c9e262b9e None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:03.206200 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 655/1281] 10.222.0.81 () {56 vars in 1143 bytes} [Mon Sep 19 11:14:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:14:03.211825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65232f03-2a7b-4cc3-b6b5-d93b434eaa16 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:03.302792 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:14:03.313579 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65232f03-2a7b-4cc3-b6b5-d93b434eaa16 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:03.346304 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2e83f-ba8a-4baa-8ac3-da6ffeb54f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:03.346906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2e83f-ba8a-4baa-8ac3-da6ffeb54f06 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:03.347186 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2e83f-ba8a-4baa-8ac3-da6ffeb54f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:03.347457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2e83f-ba8a-4baa-8ac3-da6ffeb54f06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:03.412201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea2e83f-ba8a-4baa-8ac3-da6ffeb54f06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:03.413582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea2e83f-ba8a-4baa-8ac3-da6ffeb54f06 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:03.498205 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:14:03.509164 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 627/1282] 10.222.0.81 () {58 vars in 1282 bytes} [Mon Sep 19 11:14:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 11:14:03.515143 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68508ca1-c872-431d-97d7-7b1c9e262b9e None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:03.574317 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:14:03.612086 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68508ca1-c872-431d-97d7-7b1c9e262b9e None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:03.643884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a95a3-e0ca-47db-b87a-a5fe02748c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:03.644521 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a95a3-e0ca-47db-b87a-a5fe02748c50 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:03.644797 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a95a3-e0ca-47db-b87a-a5fe02748c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:03.645068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a95a3-e0ca-47db-b87a-a5fe02748c50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:03.655203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a95a3-e0ca-47db-b87a-a5fe02748c50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:03.656659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a95a3-e0ca-47db-b87a-a5fe02748c50 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:03.671723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-fc7a95a3-e0ca-47db-b87a-a5fe02748c50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:14:03.671723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:14:03.671723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:14:03.671723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:14:03.671723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:14:03.671723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:14:03.671723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:14:03.671723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:14:03.671723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:14:03.671723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:14:03.671723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:14:03.671723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:14:03.671723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 11:14:03.671723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 11:14:03.671723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 11:14:03.671723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 11:14:03.676280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:14:03.676280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:14:03.676280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:14:03.676280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:14:03.676280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:14:03.676280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:14:03.676280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:14:03.676280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:14:03.676280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:14:03.676280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:14:03.676280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:14:03.676280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:14:03.676280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:14:03.676280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:14:03.676280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:14:03.676280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:14:03.676280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 11:14:03.677316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 11:14:03.677316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 11:14:03.677316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:14:03.677316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 656/1283] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:14:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:14:03.678034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2e83f-ba8a-4baa-8ac3-da6ffeb54f06 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:03.773070 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2e83f-ba8a-4baa-8ac3-da6ffeb54f06 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:03.806026 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4af5cf-2cb5-40e2-b554-edaa456c0635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:03.806620 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4af5cf-2cb5-40e2-b554-edaa456c0635 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:03.806898 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4af5cf-2cb5-40e2-b554-edaa456c0635 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:03.807167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4af5cf-2cb5-40e2-b554-edaa456c0635 None admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:03.807822 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4af5cf-2cb5-40e2-b554-edaa456c0635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:14:03.808126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4af5cf-2cb5-40e2-b554-edaa456c0635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:03.809507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4af5cf-2cb5-40e2-b554-edaa456c0635 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:03.820413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 628/1284] 10.222.0.81 () {56 vars in 1109 bytes} [Mon Sep 19 11:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:14:03.826844 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a95a3-e0ca-47db-b87a-a5fe02748c50 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:03.924851 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a95a3-e0ca-47db-b87a-a5fe02748c50 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:03.937872 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:14:03.958591 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b60bd8-063a-4fbc-97c3-d92b822d9c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:03.959208 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b60bd8-063a-4fbc-97c3-d92b822d9c14 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:03.959488 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b60bd8-063a-4fbc-97c3-d92b822d9c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:03.959780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b60bd8-063a-4fbc-97c3-d92b822d9c14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:03.966751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b60bd8-063a-4fbc-97c3-d92b822d9c14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:03.968211 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b60bd8-063a-4fbc-97c3-d92b822d9c14 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:03.979141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application [None req-f5b60bd8-063a-4fbc-97c3-d92b822d9c14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:14:03.979141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:14:03.979141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:14:03.979141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:14:03.979141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:14:03.979141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:14:03.979141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:14:03.979141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:14:03.979141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:14:03.979141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:14:03.979141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:14:03.979141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:14:03.979141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 11:14:03.979141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 11:14:03.979141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 11:14:03.980209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 11:14:03.980209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:14:03.980209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:14:03.980209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:14:03.980209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:14:03.980209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:14:03.980209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:14:03.980209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:14:03.980209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application with Lock( Sep 19 11:14:03.980209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:14:03.980209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:14:03.980209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:14:03.980209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:14:03.980209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:14:03.980209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:14:03.980209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:14:03.980209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:14:03.981474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 11:14:03.981474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 11:14:03.981474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 11:14:03.981474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 11:14:03.981474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 11:14:03.981474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 11:14:03.981474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 11:14:03.981474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: ERROR keystone.server.flask.application Sep 19 11:14:03.981474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 657/1285] 10.222.0.81 () {56 vars in 1111 bytes} [Mon Sep 19 11:14:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:14:03.986147 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4af5cf-2cb5-40e2-b554-edaa456c0635 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:04.034667 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:14:04.082724 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4af5cf-2cb5-40e2-b554-edaa456c0635 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:04.119404 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2c4fe-c7b1-4344-9855-68e3abd2bd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:04.120028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2c4fe-c7b1-4344-9855-68e3abd2bd33 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:04.120310 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2c4fe-c7b1-4344-9855-68e3abd2bd33 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:04.120583 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2c4fe-c7b1-4344-9855-68e3abd2bd33 None admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:04.121271 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2c4fe-c7b1-4344-9855-68e3abd2bd33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:14:04.121576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2c4fe-c7b1-4344-9855-68e3abd2bd33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:04.123000 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2c4fe-c7b1-4344-9855-68e3abd2bd33 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:04.134486 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 629/1286] 10.222.0.81 () {56 vars in 1148 bytes} [Mon Sep 19 11:14:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:14:04.141057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b60bd8-063a-4fbc-97c3-d92b822d9c14 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:04.237808 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b60bd8-063a-4fbc-97c3-d92b822d9c14 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:04.270528 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318734a-d81b-4ca6-b505-9627ad58a52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:04.271110 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318734a-d81b-4ca6-b505-9627ad58a52b None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:04.271390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318734a-d81b-4ca6-b505-9627ad58a52b None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:04.271662 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318734a-d81b-4ca6-b505-9627ad58a52b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:04.272408 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318734a-d81b-4ca6-b505-9627ad58a52b None admin] RBAC: Adding query filter params (role.id=753f8b35d5f0458d8f62d0ad6e28033f, scope.project.id=ed9a0b626e3e4b93b391158e981ec586, user.id=6e7a18dc4a4d4d35a05f1aafbaaf9c31) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:14:04.272714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318734a-d81b-4ca6-b505-9627ad58a52b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:04.274119 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318734a-d81b-4ca6-b505-9627ad58a52b None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:04.279684 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 658/1287] 10.222.0.81 () {56 vars in 1374 bytes} [Mon Sep 19 11:14:04 2022] GET /identity/v3/role_assignments?user.id=6e7a18dc4a4d4d35a05f1aafbaaf9c31&scope.project.id=ed9a0b626e3e4b93b391158e981ec586&role.id=753f8b35d5f0458d8f62d0ad6e28033f => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:14:06.422664 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087c143-9be4-46d5-8ec2-e523dc791160 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:06.423229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087c143-9be4-46d5-8ec2-e523dc791160 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:06.423229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087c143-9be4-46d5-8ec2-e523dc791160 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:06.424599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 630/1288] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:14:06.429673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c824b71-3c00-483d-9bc6-09bd7480a9df None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:06.430062 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c824b71-3c00-483d-9bc6-09bd7480a9df None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:06.430238 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c824b71-3c00-483d-9bc6-09bd7480a9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:06.448372 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-6c824b71-3c00-483d-9bc6-09bd7480a9df None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:14:06.449558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-6c824b71-3c00-483d-9bc6-09bd7480a9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:14:06.478305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:14:06.516125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 659/1289] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:14:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:14:06.791295 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087c143-9be4-46d5-8ec2-e523dc791160 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:06.889706 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087c143-9be4-46d5-8ec2-e523dc791160 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:06.899986 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:14:06.923792 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f689984-32f4-4123-8611-b6f0d1b35007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:06.924402 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f689984-32f4-4123-8611-b6f0d1b35007 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:06.924678 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f689984-32f4-4123-8611-b6f0d1b35007 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:06.924948 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f689984-32f4-4123-8611-b6f0d1b35007 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:06.925517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f689984-32f4-4123-8611-b6f0d1b35007 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:06.926880 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f689984-32f4-4123-8611-b6f0d1b35007 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:06.935303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-6f689984-32f4-4123-8611-b6f0d1b35007 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:14:06.935303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:14:06.935303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:14:06.935303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:14:06.935303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:14:06.935303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:14:06.935303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:14:06.935303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:14:06.935303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:14:06.935303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:14:06.935303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:14:06.935303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:14:06.935303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:14:06.935303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:14:06.935303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:14:06.935303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:14:06.935303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:14:06.936545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:14:06.936545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:14:06.936545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:14:06.936545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:14:06.936545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:14:06.936545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:14:06.936545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:14:06.936545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:14:06.936545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:14:06.936545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:14:06.936545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:14:06.936545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:14:06.936545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:14:06.936545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:14:06.936545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:14:06.936545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:14:06.937806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:14:06.937806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:14:06.937806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:14:06.937806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:14:06.937806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:14:06.937806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:14:06.937806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:14:06.937806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:14:06.937806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:14:06.937806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:14:06.937806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:14:06.937806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:14:06.937806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:14:06.937806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:14:06.937806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 631/1290] 10.222.0.81 () {56 vars in 1117 bytes} [Mon Sep 19 11:14:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:14:06.942318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c824b71-3c00-483d-9bc6-09bd7480a9df None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:07.037642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c824b71-3c00-483d-9bc6-09bd7480a9df None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:07.069952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddbecf-f468-48a0-864d-684268dbb955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:07.070481 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddbecf-f468-48a0-864d-684268dbb955 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:07.070759 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddbecf-f468-48a0-864d-684268dbb955 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:07.071031 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddbecf-f468-48a0-864d-684268dbb955 None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:07.071656 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddbecf-f468-48a0-864d-684268dbb955 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:14:07.071988 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddbecf-f468-48a0-864d-684268dbb955 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:07.073610 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddbecf-f468-48a0-864d-684268dbb955 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:07.078809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 660/1291] 10.222.0.81 () {56 vars in 1116 bytes} [Mon Sep 19 11:14:06 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:14:07.084562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f689984-32f4-4123-8611-b6f0d1b35007 None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:07.181067 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f689984-32f4-4123-8611-b6f0d1b35007 None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:07.213900 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaeed7f-2038-4e67-817d-04d708aaec25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:07.214479 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaeed7f-2038-4e67-817d-04d708aaec25 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:07.214759 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaeed7f-2038-4e67-817d-04d708aaec25 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:07.215032 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaeed7f-2038-4e67-817d-04d708aaec25 None admin] PATH_INFO: `/v3/services` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:07.216207 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaeed7f-2038-4e67-817d-04d708aaec25 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:14:07.216506 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaeed7f-2038-4e67-817d-04d708aaec25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:07.217861 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaeed7f-2038-4e67-817d-04d708aaec25 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:07.222127 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 632/1292] 10.222.0.81 () {56 vars in 1116 bytes} [Mon Sep 19 11:14:07 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:14:09.414006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba05792-691f-4586-9409-147b1574f31f None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:09.414566 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba05792-691f-4586-9409-147b1574f31f None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:09.414566 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba05792-691f-4586-9409-147b1574f31f None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:09.416228 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 661/1293] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:14:09.422513 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7c94f-6285-4698-b647-7537bbe37d6f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:09.423009 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7c94f-6285-4698-b647-7537bbe37d6f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:09.423176 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7c94f-6285-4698-b647-7537bbe37d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:09.442863 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-bdb7c94f-6285-4698-b647-7537bbe37d6f None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:14:09.444088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bdb7c94f-6285-4698-b647-7537bbe37d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:14:09.473832 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:14:09.512770 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 633/1294] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:14:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 11:14:09.775870 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba05792-691f-4586-9409-147b1574f31f None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:09.806414 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:14:09.884054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba05792-691f-4586-9409-147b1574f31f None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:09.916848 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcafdd-1f25-4cca-b60b-2405e6d5d923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:09.917435 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcafdd-1f25-4cca-b60b-2405e6d5d923 None admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:09.917708 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcafdd-1f25-4cca-b60b-2405e6d5d923 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:09.917971 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcafdd-1f25-4cca-b60b-2405e6d5d923 None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:09.919092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fcafdd-1f25-4cca-b60b-2405e6d5d923 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:09.920444 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fcafdd-1f25-4cca-b60b-2405e6d5d923 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:09.934969 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 662/1295] 10.222.0.81 () {60 vars in 1141 bytes} [Mon Sep 19 11:14:09 2022] POST /identity/v3/services => generated 246 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:14:12.088945 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10a6aa-da24-4d5a-a8bf-5b43f398e16f None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:12.089503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10a6aa-da24-4d5a-a8bf-5b43f398e16f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:12.089503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10a6aa-da24-4d5a-a8bf-5b43f398e16f None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:12.090874 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 634/1296] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:14:12.096017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae1ee3-4faf-420b-b2a9-9d03731fd781 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:12.096432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae1ee3-4faf-420b-b2a9-9d03731fd781 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:12.096569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae1ee3-4faf-420b-b2a9-9d03731fd781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:12.114973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-0eae1ee3-4faf-420b-b2a9-9d03731fd781 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:14:12.116192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-0eae1ee3-4faf-420b-b2a9-9d03731fd781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:14:12.185953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 663/1297] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:14:12 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:14:12.463695 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10a6aa-da24-4d5a-a8bf-5b43f398e16f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:12.563041 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10a6aa-da24-4d5a-a8bf-5b43f398e16f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:12.596134 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff623696-7834-474e-addb-750a68f0a41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:12.596741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff623696-7834-474e-addb-750a68f0a41b None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:12.597022 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff623696-7834-474e-addb-750a68f0a41b None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:12.597297 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff623696-7834-474e-addb-750a68f0a41b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:12.597874 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff623696-7834-474e-addb-750a68f0a41b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:12.599232 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff623696-7834-474e-addb-750a68f0a41b None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:12.607821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-ff623696-7834-474e-addb-750a68f0a41b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:14:12.607821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:14:12.607821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:14:12.607821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:14:12.607821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:14:12.607821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:14:12.607821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:14:12.607821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:14:12.607821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:14:12.607821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:14:12.607821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:14:12.607821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:14:12.607821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:14:12.607821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:14:12.607821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:14:12.607821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:14:12.607821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:14:12.609006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:14:12.609006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:14:12.609006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:14:12.609006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:14:12.609006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:14:12.609006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:14:12.609006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:14:12.609006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:14:12.609006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:14:12.609006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:14:12.609006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:14:12.609006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:14:12.609006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:14:12.609006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:14:12.609006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:14:12.609006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:14:12.610044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:14:12.610044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:14:12.610044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:14:12.610044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:14:12.610044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:14:12.610044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:14:12.610044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:14:12.610044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:14:12.610044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:14:12.610044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:14:12.610044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:14:12.610044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:14:12.610044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:14:12.610044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:14:12.610044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 635/1298] 10.222.0.81 () {56 vars in 1117 bytes} [Mon Sep 19 11:14:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:14:12.614846 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae1ee3-4faf-420b-b2a9-9d03731fd781 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:12.632875 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:14:12.713653 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae1ee3-4faf-420b-b2a9-9d03731fd781 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:12.745602 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873824d-6867-4df5-8afb-6985fc595c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:12.746634 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873824d-6867-4df5-8afb-6985fc595c66 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:12.746910 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873824d-6867-4df5-8afb-6985fc595c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:12.747178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873824d-6867-4df5-8afb-6985fc595c66 None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:12.747842 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873824d-6867-4df5-8afb-6985fc595c66 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:14:12.748149 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873824d-6867-4df5-8afb-6985fc595c66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:12.749500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873824d-6867-4df5-8afb-6985fc595c66 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:12.754516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 664/1299] 10.222.0.81 () {56 vars in 1116 bytes} [Mon Sep 19 11:14:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:14:12.760628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff623696-7834-474e-addb-750a68f0a41b None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:12.791414 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:14:12.858554 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff623696-7834-474e-addb-750a68f0a41b None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:12.891533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6681b-09a6-4f2b-8167-4c4c88742137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:12.892148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6681b-09a6-4f2b-8167-4c4c88742137 None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:12.892433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6681b-09a6-4f2b-8167-4c4c88742137 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:12.892709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6681b-09a6-4f2b-8167-4c4c88742137 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:12.893403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec6681b-09a6-4f2b-8167-4c4c88742137 None admin] RBAC: Adding query filter params (interface=public, service_id=d4d4103b6e914efa82bdb6a5fc41bd20, region_id=RegionOne) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:14:12.893711 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec6681b-09a6-4f2b-8167-4c4c88742137 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:12.895072 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec6681b-09a6-4f2b-8167-4c4c88742137 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:12.899906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 636/1300] 10.222.0.81 () {56 vars in 1251 bytes} [Mon Sep 19 11:14:12 2022] GET /identity/v3/endpoints?service_id=d4d4103b6e914efa82bdb6a5fc41bd20&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:14:12.906002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873824d-6867-4df5-8afb-6985fc595c66 None admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:12.985211 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:14:13.002959 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873824d-6867-4df5-8afb-6985fc595c66 None admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:13.036062 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c22ca-109f-4d92-a9bd-309ef5329d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:13.036652 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c22ca-109f-4d92-a9bd-309ef5329d22 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:13.036931 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c22ca-109f-4d92-a9bd-309ef5329d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:13.037204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c22ca-109f-4d92-a9bd-309ef5329d22 None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:13.037801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c22ca-109f-4d92-a9bd-309ef5329d22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:13.039154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c22ca-109f-4d92-a9bd-309ef5329d22 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:13.045791 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 665/1301] 10.222.0.81 () {56 vars in 1087 bytes} [Mon Sep 19 11:14:12 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:14:15.174427 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f16a8d-23d2-4176-badf-1250ccd30d9f None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:15.174997 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f16a8d-23d2-4176-badf-1250ccd30d9f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:15.174997 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f16a8d-23d2-4176-badf-1250ccd30d9f None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:15.176355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 637/1302] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:14:15.181564 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3a090-8d1d-405c-a528-f62b5bc219ae None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:15.181961 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3a090-8d1d-405c-a528-f62b5bc219ae None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:15.182115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3a090-8d1d-405c-a528-f62b5bc219ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:15.201625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-52b3a090-8d1d-405c-a528-f62b5bc219ae None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:14:15.202808 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-52b3a090-8d1d-405c-a528-f62b5bc219ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:14:15.267555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 666/1303] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:14:15 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 11:14:15.532710 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f16a8d-23d2-4176-badf-1250ccd30d9f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:15.624421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:14:15.631200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f16a8d-23d2-4176-badf-1250ccd30d9f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:15.664819 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d4c27-5998-4c5e-9d41-be89b15615ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:15.665436 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d4c27-5998-4c5e-9d41-be89b15615ae None admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:15.665714 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d4c27-5998-4c5e-9d41-be89b15615ae None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:15.665981 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d4c27-5998-4c5e-9d41-be89b15615ae None admin] PATH_INFO: `/v3/services/placement` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:15.666583 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d4c27-5998-4c5e-9d41-be89b15615ae None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:15.667995 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d4c27-5998-4c5e-9d41-be89b15615ae None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:15.676679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application [None req-d52d4c27-5998-4c5e-9d41-be89b15615ae None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:14:15.676679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:14:15.676679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 11:14:15.676679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 11:14:15.676679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 11:14:15.676679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 11:14:15.676679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 11:14:15.676679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:14:15.676679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:14:15.676679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:14:15.676679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 11:14:15.676679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:14:15.676679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 11:14:15.676679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 11:14:15.676679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 11:14:15.676679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 11:14:15.676679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 11:14:15.677956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 11:14:15.677956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 11:14:15.677956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 11:14:15.677956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 11:14:15.677956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 11:14:15.677956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 11:14:15.677956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 11:14:15.677956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 11:14:15.677956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 11:14:15.677956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 11:14:15.677956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 11:14:15.677956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 11:14:15.677956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 11:14:15.677956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 11:14:15.677956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 11:14:15.677956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 11:14:15.679082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 11:14:15.679082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application with Lock( Sep 19 11:14:15.679082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 11:14:15.679082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self._enter() Sep 19 11:14:15.679082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 11:14:15.679082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 11:14:15.679082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 11:14:15.679082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application return self.creator() Sep 19 11:14:15.679082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 11:14:15.679082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application created_value = creator( Sep 19 11:14:15.679082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 11:14:15.679082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 11:14:15.679082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 11:14:15.679082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: ERROR keystone.server.flask.application Sep 19 11:14:15.679082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 638/1304] 10.222.0.81 () {56 vars in 1117 bytes} [Mon Sep 19 11:14:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 11:14:15.684293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3a090-8d1d-405c-a528-f62b5bc219ae None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:15.758100 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:14:15.781062 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3a090-8d1d-405c-a528-f62b5bc219ae None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:15.814097 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc13a31-a2d6-4c85-b39d-d11cd1bb4a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:15.814629 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc13a31-a2d6-4c85-b39d-d11cd1bb4a21 None admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:15.814905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc13a31-a2d6-4c85-b39d-d11cd1bb4a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:15.815176 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc13a31-a2d6-4c85-b39d-d11cd1bb4a21 None admin] PATH_INFO: `/v3/services` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:15.815847 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc13a31-a2d6-4c85-b39d-d11cd1bb4a21 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:14:15.816147 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc13a31-a2d6-4c85-b39d-d11cd1bb4a21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:15.817526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc13a31-a2d6-4c85-b39d-d11cd1bb4a21 None admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:15.822685 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 667/1305] 10.222.0.81 () {56 vars in 1116 bytes} [Mon Sep 19 11:14:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:14:15.828379 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d4c27-5998-4c5e-9d41-be89b15615ae None admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:14:15.924411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d4c27-5998-4c5e-9d41-be89b15615ae None admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:14:15.957634 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ffed2b-f4ee-4bde-bc0b-7ca34664b2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:14:15.958215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffed2b-f4ee-4bde-bc0b-7ca34664b2b6 None admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:14:15.958496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffed2b-f4ee-4bde-bc0b-7ca34664b2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:14:15.958767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffed2b-f4ee-4bde-bc0b-7ca34664b2b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:14:15.959436 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ffed2b-f4ee-4bde-bc0b-7ca34664b2b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:14:15.960829 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ffed2b-f4ee-4bde-bc0b-7ca34664b2b6 None admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:14:15.980066 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 639/1306] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:14:15 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:14:25.820002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=30 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:14:25.978848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:19.734150 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8203b-a977-44ec-9956-2701a861ff28 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:19.734608 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8203b-a977-44ec-9956-2701a861ff28 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:19.734608 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8203b-a977-44ec-9956-2701a861ff28 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:19.742248 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe940e2-69d7-46c2-b470-2a29bf53908d None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:19.742878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe940e2-69d7-46c2-b470-2a29bf53908d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:19.742878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe940e2-69d7-46c2-b470-2a29bf53908d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:19.747946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 668/1307] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:15:19 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:15:19.842530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-cbe940e2-69d7-46c2-b470-2a29bf53908d None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:15:19.843860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cbe940e2-69d7-46c2-b470-2a29bf53908d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:19.950639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 640/1308] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:15:20.251061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e100006-fbad-4a05-8ba0-5a1f26f23ef1 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:20.251345 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e100006-fbad-4a05-8ba0-5a1f26f23ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:20.251812 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e100006-fbad-4a05-8ba0-5a1f26f23ef1 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:20.253414 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 669/1309] 10.222.0.81 () {54 vars in 963 bytes} [Mon Sep 19 11:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:15:20.260706 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21498d-d29f-4fe3-919a-6f397f8986c6 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:20.261147 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21498d-d29f-4fe3-919a-6f397f8986c6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:20.261500 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21498d-d29f-4fe3-919a-6f397f8986c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:20.350868 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9d21498d-d29f-4fe3-919a-6f397f8986c6 None None] MFA Rules not processed for user `65de2e5f67dd4bb6a20cf3934afc8abd`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:15:20.352190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9d21498d-d29f-4fe3-919a-6f397f8986c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:20.374507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:20.437057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 641/1310] 10.222.0.81 () {58 vars in 1060 bytes} [Mon Sep 19 11:15:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:15:20.447045 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e100006-fbad-4a05-8ba0-5a1f26f23ef1 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:15:20.448857 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-1e100006-fbad-4a05-8ba0-5a1f26f23ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:20.572448 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e100006-fbad-4a05-8ba0-5a1f26f23ef1 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:15:20.658545 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec0732-2ebf-4830-881e-2028400b7983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:15:20.659216 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec0732-2ebf-4830-881e-2028400b7983 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:20.659501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec0732-2ebf-4830-881e-2028400b7983 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:20.659791 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec0732-2ebf-4830-881e-2028400b7983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:20.661551 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-d8ec0732-2ebf-4830-881e-2028400b7983 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:20.721543 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:20.782324 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec0732-2ebf-4830-881e-2028400b7983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:15:20.783805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec0732-2ebf-4830-881e-2028400b7983 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:15:20.806823 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 670/1311] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:15:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:15:23.267528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0f7c8-1b0c-454b-8d1a-576427fcd9a0 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:23.268091 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0f7c8-1b0c-454b-8d1a-576427fcd9a0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:23.268091 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0f7c8-1b0c-454b-8d1a-576427fcd9a0 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:23.269362 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 642/1312] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:15:23.274797 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc112b-33c0-4bb4-a28f-5c406785fb73 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:23.275201 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc112b-33c0-4bb4-a28f-5c406785fb73 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:23.275481 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc112b-33c0-4bb4-a28f-5c406785fb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:23.360391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-c4fc112b-33c0-4bb4-a28f-5c406785fb73 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:15:23.361616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c4fc112b-33c0-4bb4-a28f-5c406785fb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:23.461018 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 671/1313] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:15:23.757971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4a25e-b5cb-4a44-bd6d-16d8d1e9366d None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:23.758258 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4a25e-b5cb-4a44-bd6d-16d8d1e9366d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:23.758531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4a25e-b5cb-4a44-bd6d-16d8d1e9366d None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:23.759920 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 643/1314] 10.222.0.81 () {54 vars in 963 bytes} [Mon Sep 19 11:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:15:23.765762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd79eba-9517-44a2-8768-01c4b2baf089 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:23.766046 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd79eba-9517-44a2-8768-01c4b2baf089 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:23.766319 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd79eba-9517-44a2-8768-01c4b2baf089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:23.778312 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:23.842506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-cfd79eba-9517-44a2-8768-01c4b2baf089 None None] MFA Rules not processed for user `65de2e5f67dd4bb6a20cf3934afc8abd`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:15:23.843729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-cfd79eba-9517-44a2-8768-01c4b2baf089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:23.918030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 672/1315] 10.222.0.81 () {58 vars in 1060 bytes} [Mon Sep 19 11:15:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:15:23.927082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4a25e-b5cb-4a44-bd6d-16d8d1e9366d None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:15:23.928671 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9ad4a25e-b5cb-4a44-bd6d-16d8d1e9366d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:24.030590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4a25e-b5cb-4a44-bd6d-16d8d1e9366d None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:15:24.099305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:24.102377 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ec852-28d1-4ae9-b2c1-d5009050970d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:15:24.103170 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ec852-28d1-4ae9-b2c1-d5009050970d service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:24.103653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ec852-28d1-4ae9-b2c1-d5009050970d service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:24.103975 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ec852-28d1-4ae9-b2c1-d5009050970d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:24.105767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e41ec852-28d1-4ae9-b2c1-d5009050970d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:24.210941 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ec852-28d1-4ae9-b2c1-d5009050970d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:15:24.212338 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ec852-28d1-4ae9-b2c1-d5009050970d service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:15:24.230037 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 644/1316] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:15:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:15:26.928421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5a979-2506-47d5-885e-3a11274a6cb0 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:26.928950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5a979-2506-47d5-885e-3a11274a6cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:26.928950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5a979-2506-47d5-885e-3a11274a6cb0 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:26.930219 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 673/1317] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:15:26.935475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45ec60-9c76-463e-b21f-da0c0482f7d2 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:26.935948 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45ec60-9c76-463e-b21f-da0c0482f7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:26.936046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45ec60-9c76-463e-b21f-da0c0482f7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:27.001842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9a45ec60-9c76-463e-b21f-da0c0482f7d2 None None] MFA Rules not processed for user `6aeda366bbb94f7cb91aa6788d9285e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:15:27.003076 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9a45ec60-9c76-463e-b21f-da0c0482f7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:27.054869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:27.099631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 645/1318] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:15:26 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:15:27.384101 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5a979-2506-47d5-885e-3a11274a6cb0 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:15:27.385641 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f5c5a979-2506-47d5-885e-3a11274a6cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:27.487935 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5a979-2506-47d5-885e-3a11274a6cb0 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:15:27.490763 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:27.561938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31674fee-a797-4f33-9c07-506a7035d5e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:15:27.562549 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31674fee-a797-4f33-9c07-506a7035d5e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:27.562897 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31674fee-a797-4f33-9c07-506a7035d5e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:27.563167 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31674fee-a797-4f33-9c07-506a7035d5e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:27.564782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-31674fee-a797-4f33-9c07-506a7035d5e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:27.678736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31674fee-a797-4f33-9c07-506a7035d5e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:15:27.680298 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31674fee-a797-4f33-9c07-506a7035d5e0 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:15:27.698607 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 674/1319] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:15:27 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:15:30.324675 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78eaa4-728d-47ab-a28d-10f43596dd3b None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:30.325219 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78eaa4-728d-47ab-a28d-10f43596dd3b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:30.325219 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78eaa4-728d-47ab-a28d-10f43596dd3b None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:30.326545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 646/1320] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:15:30.331940 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4bcf2-f706-4d6d-903c-8d6e65125255 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:30.332401 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4bcf2-f706-4d6d-903c-8d6e65125255 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:30.332514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4bcf2-f706-4d6d-903c-8d6e65125255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:30.373263 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:30.399250 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-42d4bcf2-f706-4d6d-903c-8d6e65125255 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:15:30.400561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-42d4bcf2-f706-4d6d-903c-8d6e65125255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:30.494180 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 675/1321] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:15:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:15:30.775436 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78eaa4-728d-47ab-a28d-10f43596dd3b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:15:30.777060 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3d78eaa4-728d-47ab-a28d-10f43596dd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:30.877033 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78eaa4-728d-47ab-a28d-10f43596dd3b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:15:30.949370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75feee-17e5-42e2-8d03-988e9e4471fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:15:30.949898 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75feee-17e5-42e2-8d03-988e9e4471fa service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:30.950176 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75feee-17e5-42e2-8d03-988e9e4471fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:30.950444 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75feee-17e5-42e2-8d03-988e9e4471fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:30.952054 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3b75feee-17e5-42e2-8d03-988e9e4471fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:30.959238 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:31.058954 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b75feee-17e5-42e2-8d03-988e9e4471fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:15:31.060454 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b75feee-17e5-42e2-8d03-988e9e4471fa service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:15:31.078437 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 647/1322] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:15:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:15:33.865892 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71aa796-b11b-42af-b567-5844061ece66 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:33.866455 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71aa796-b11b-42af-b567-5844061ece66 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:33.866455 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71aa796-b11b-42af-b567-5844061ece66 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:33.867800 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 676/1323] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:15:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:15:33.873151 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fb25b-8336-40bd-8bd0-9bb86e355e62 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:33.873587 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fb25b-8336-40bd-8bd0-9bb86e355e62 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:33.873683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fb25b-8336-40bd-8bd0-9bb86e355e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:33.942608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-661fb25b-8336-40bd-8bd0-9bb86e355e62 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:15:33.944141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-661fb25b-8336-40bd-8bd0-9bb86e355e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:34.007833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:34.049882 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 648/1324] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:15:34.338784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71aa796-b11b-42af-b567-5844061ece66 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:15:34.340425 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f71aa796-b11b-42af-b567-5844061ece66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:34.446816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71aa796-b11b-42af-b567-5844061ece66 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:15:34.459817 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:34.522329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979179c6-962d-4b6c-b859-ca32254941eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:15:34.522942 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979179c6-962d-4b6c-b859-ca32254941eb service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:34.523222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979179c6-962d-4b6c-b859-ca32254941eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:34.523492 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979179c6-962d-4b6c-b859-ca32254941eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:34.525110 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-979179c6-962d-4b6c-b859-ca32254941eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:34.634786 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979179c6-962d-4b6c-b859-ca32254941eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:15:34.636259 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979179c6-962d-4b6c-b859-ca32254941eb service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:15:34.655188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 677/1325] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:15:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:15:36.869727 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfdad9-ef2b-4e5d-8f45-4ee5f2e23e34 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:36.870265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfdad9-ef2b-4e5d-8f45-4ee5f2e23e34 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:36.870265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfdad9-ef2b-4e5d-8f45-4ee5f2e23e34 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:36.871629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 649/1326] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:15:36.876987 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf984f-30d7-4ef6-8a51-c398c97b3447 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:36.877428 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf984f-30d7-4ef6-8a51-c398c97b3447 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:36.877570 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf984f-30d7-4ef6-8a51-c398c97b3447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:36.925367 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:36.945175 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-c1cf984f-30d7-4ef6-8a51-c398c97b3447 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:15:36.946384 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c1cf984f-30d7-4ef6-8a51-c398c97b3447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:37.051582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 678/1327] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:15:37.339321 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cfdad9-ef2b-4e5d-8f45-4ee5f2e23e34 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:15:37.340909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-29cfdad9-ef2b-4e5d-8f45-4ee5f2e23e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:37.444444 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cfdad9-ef2b-4e5d-8f45-4ee5f2e23e34 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:15:37.520886 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c1d7e-8550-4677-9026-5d5287456c24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:15:37.521493 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c1d7e-8550-4677-9026-5d5287456c24 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:37.521775 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c1d7e-8550-4677-9026-5d5287456c24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:37.522044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c1d7e-8550-4677-9026-5d5287456c24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:37.523640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d99c1d7e-8550-4677-9026-5d5287456c24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:37.543911 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:37.636099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99c1d7e-8550-4677-9026-5d5287456c24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:15:37.637545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99c1d7e-8550-4677-9026-5d5287456c24 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:15:37.658546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 650/1328] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:15:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:15:39.847654 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8aa302-0a6b-4b40-8fe6-31bf686a563a None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:39.848186 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8aa302-0a6b-4b40-8fe6-31bf686a563a None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:39.848186 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8aa302-0a6b-4b40-8fe6-31bf686a563a None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:39.849675 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 679/1329] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:15:39.855122 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53533c6b-7f54-458e-8511-8de9206a26d0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:39.855540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53533c6b-7f54-458e-8511-8de9206a26d0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:39.855672 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53533c6b-7f54-458e-8511-8de9206a26d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:39.921119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-53533c6b-7f54-458e-8511-8de9206a26d0 None None] MFA Rules not processed for user `6aeda366bbb94f7cb91aa6788d9285e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:15:39.922671 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-53533c6b-7f54-458e-8511-8de9206a26d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:39.987901 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:40.021449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 651/1330] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:15:39 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:15:40.308073 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8aa302-0a6b-4b40-8fe6-31bf686a563a None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:15:40.309975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-8b8aa302-0a6b-4b40-8fe6-31bf686a563a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:40.417283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8aa302-0a6b-4b40-8fe6-31bf686a563a None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:15:40.438974 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:40.494168 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69389e8c-b8ed-4762-ab1a-8c608f6aa0ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:15:40.494771 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69389e8c-b8ed-4762-ab1a-8c608f6aa0ae service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:40.495047 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69389e8c-b8ed-4762-ab1a-8c608f6aa0ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:40.495324 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69389e8c-b8ed-4762-ab1a-8c608f6aa0ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:40.496958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-69389e8c-b8ed-4762-ab1a-8c608f6aa0ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:40.610546 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69389e8c-b8ed-4762-ab1a-8c608f6aa0ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:15:40.612662 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69389e8c-b8ed-4762-ab1a-8c608f6aa0ae service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:15:40.633642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 680/1331] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:15:40 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:15:43.012349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc55bd-528d-4bb4-a6c7-49cb47595fa0 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:43.012759 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc55bd-528d-4bb4-a6c7-49cb47595fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:43.013025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc55bd-528d-4bb4-a6c7-49cb47595fa0 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:43.014477 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 652/1332] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:15:43.019607 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a430a5-bbae-4efb-ad2a-1946ea98dfdf None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:43.020776 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a430a5-bbae-4efb-ad2a-1946ea98dfdf None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:43.021057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a430a5-bbae-4efb-ad2a-1946ea98dfdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:43.087134 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:43.090207 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-e5a430a5-bbae-4efb-ad2a-1946ea98dfdf None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:15:43.092065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e5a430a5-bbae-4efb-ad2a-1946ea98dfdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:43.191794 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 681/1333] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:15:43.469847 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc55bd-528d-4bb4-a6c7-49cb47595fa0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:15:43.471391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9ecc55bd-528d-4bb4-a6c7-49cb47595fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:43.573748 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc55bd-528d-4bb4-a6c7-49cb47595fa0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:15:43.644815 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c618f-80c9-41b1-8d13-105b57fd2a9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:15:43.645459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c618f-80c9-41b1-8d13-105b57fd2a9e service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:43.645737 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c618f-80c9-41b1-8d13-105b57fd2a9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:43.646008 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c618f-80c9-41b1-8d13-105b57fd2a9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:43.647588 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-159c618f-80c9-41b1-8d13-105b57fd2a9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:43.677220 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:43.759016 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159c618f-80c9-41b1-8d13-105b57fd2a9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:15:43.760423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159c618f-80c9-41b1-8d13-105b57fd2a9e service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:15:43.778143 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 653/1334] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:15:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:15:45.963514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f02855-226d-406a-9d1c-34e21cf58cc8 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:45.964061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f02855-226d-406a-9d1c-34e21cf58cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:45.964061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f02855-226d-406a-9d1c-34e21cf58cc8 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:45.965347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 682/1335] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:15:45.970833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5cb95-bf9a-4930-b2e6-e38120550329 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:45.971268 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5cb95-bf9a-4930-b2e6-e38120550329 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:45.971387 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5cb95-bf9a-4930-b2e6-e38120550329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:46.036157 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-89b5cb95-bf9a-4930-b2e6-e38120550329 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:15:46.037362 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-89b5cb95-bf9a-4930-b2e6-e38120550329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:46.110347 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:46.132584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 654/1336] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:15:46.421707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f02855-226d-406a-9d1c-34e21cf58cc8 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:15:46.423258 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-37f02855-226d-406a-9d1c-34e21cf58cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:46.526303 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f02855-226d-406a-9d1c-34e21cf58cc8 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:15:46.555383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:46.602265 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e45ebb-f647-439b-8f76-e905960090d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:15:46.602859 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e45ebb-f647-439b-8f76-e905960090d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:46.603135 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e45ebb-f647-439b-8f76-e905960090d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:46.603403 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e45ebb-f647-439b-8f76-e905960090d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:46.605008 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-14e45ebb-f647-439b-8f76-e905960090d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:46.716907 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e45ebb-f647-439b-8f76-e905960090d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:15:46.718345 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e45ebb-f647-439b-8f76-e905960090d0 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:15:46.736352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 683/1337] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:15:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:15:49.202157 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088dd2e-4cfd-4c9c-8b4d-7ff9bff8482a None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:49.202698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088dd2e-4cfd-4c9c-8b4d-7ff9bff8482a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:49.202698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088dd2e-4cfd-4c9c-8b4d-7ff9bff8482a None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:49.204106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 655/1338] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:15:49.209142 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90159448-9b1f-47db-b8e0-194da8b8f19d None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:49.209555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90159448-9b1f-47db-b8e0-194da8b8f19d None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:49.209688 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90159448-9b1f-47db-b8e0-194da8b8f19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:49.274876 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-90159448-9b1f-47db-b8e0-194da8b8f19d None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:15:49.276039 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-90159448-9b1f-47db-b8e0-194da8b8f19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:49.284102 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:49.375645 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 684/1339] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:15:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:15:49.659845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088dd2e-4cfd-4c9c-8b4d-7ff9bff8482a None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:15:49.661938 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5088dd2e-4cfd-4c9c-8b4d-7ff9bff8482a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:49.772303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088dd2e-4cfd-4c9c-8b4d-7ff9bff8482a None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:15:49.843220 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c044a9-f9cd-43f9-83c7-1ddaf64283f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:15:49.843834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c044a9-f9cd-43f9-83c7-1ddaf64283f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:49.844109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c044a9-f9cd-43f9-83c7-1ddaf64283f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:49.844375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c044a9-f9cd-43f9-83c7-1ddaf64283f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:49.845938 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-00c044a9-f9cd-43f9-83c7-1ddaf64283f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:49.881527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:49.956707 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c044a9-f9cd-43f9-83c7-1ddaf64283f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:15:49.958590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c044a9-f9cd-43f9-83c7-1ddaf64283f5 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:15:49.976783 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 656/1340] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:15:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:15:54.065963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38637e6c-eab1-4d51-902d-fd7c84331df9 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:54.067370 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38637e6c-eab1-4d51-902d-fd7c84331df9 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:54.067370 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38637e6c-eab1-4d51-902d-fd7c84331df9 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:54.071803 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 685/1341] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:15:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:15:54.074779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52687e16-4826-47ab-9707-a3c58f3670cb None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:54.075263 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52687e16-4826-47ab-9707-a3c58f3670cb None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:54.075361 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52687e16-4826-47ab-9707-a3c58f3670cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:54.149978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-52687e16-4826-47ab-9707-a3c58f3670cb None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:15:54.151209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-52687e16-4826-47ab-9707-a3c58f3670cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:54.246383 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:54.263799 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 657/1342] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:15:54.544803 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38637e6c-eab1-4d51-902d-fd7c84331df9 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:15:54.546388 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-38637e6c-eab1-4d51-902d-fd7c84331df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:54.648924 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38637e6c-eab1-4d51-902d-fd7c84331df9 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:15:54.687702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:54.727273 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fd361-80c4-4eb6-8e04-b2b48fdfb03d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:15:54.727902 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fd361-80c4-4eb6-8e04-b2b48fdfb03d service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:54.728180 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fd361-80c4-4eb6-8e04-b2b48fdfb03d service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:54.728445 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fd361-80c4-4eb6-8e04-b2b48fdfb03d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:54.730045 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5e4fd361-80c4-4eb6-8e04-b2b48fdfb03d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:54.844579 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fd361-80c4-4eb6-8e04-b2b48fdfb03d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:15:54.844579 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fd361-80c4-4eb6-8e04-b2b48fdfb03d service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:15:54.863452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 686/1343] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:15:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:15:57.707611 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d7138-fb9b-4c93-ab73-1d6471530810 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:57.708206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d7138-fb9b-4c93-ab73-1d6471530810 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:57.708206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d7138-fb9b-4c93-ab73-1d6471530810 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:57.709551 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 658/1344] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:15:57.714682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a1e01-c88c-46a6-9b79-22060c7c1444 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:57.715131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a1e01-c88c-46a6-9b79-22060c7c1444 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:57.715253 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a1e01-c88c-46a6-9b79-22060c7c1444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:57.781876 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-502a1e01-c88c-46a6-9b79-22060c7c1444 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:15:57.783088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-502a1e01-c88c-46a6-9b79-22060c7c1444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:57.801628 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:57.881249 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 687/1345] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:15:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:15:58.167713 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d7138-fb9b-4c93-ab73-1d6471530810 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:15:58.169304 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7d3d7138-fb9b-4c93-ab73-1d6471530810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:58.279400 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d7138-fb9b-4c93-ab73-1d6471530810 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:15:58.357075 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81ae04-1e3c-4d01-a43c-d2cf9d1c9501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:15:58.357705 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81ae04-1e3c-4d01-a43c-d2cf9d1c9501 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:15:58.357992 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81ae04-1e3c-4d01-a43c-d2cf9d1c9501 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:15:58.358261 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81ae04-1e3c-4d01-a43c-d2cf9d1c9501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:15:58.359877 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-dc81ae04-1e3c-4d01-a43c-d2cf9d1c9501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:15:58.404233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:15:58.475508 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81ae04-1e3c-4d01-a43c-d2cf9d1c9501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:15:58.476989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81ae04-1e3c-4d01-a43c-d2cf9d1c9501 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:15:58.495508 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 659/1346] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:15:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:02.650437 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2aad03-8750-477a-8173-a2f8d09e7e5f None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:02.650932 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2aad03-8750-477a-8173-a2f8d09e7e5f None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:02.651154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2aad03-8750-477a-8173-a2f8d09e7e5f None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:02.652450 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 688/1347] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:16:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:02.660063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d7ea1-12b9-4732-9ba5-e4a80a7d8a55 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:02.660514 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d7ea1-12b9-4732-9ba5-e4a80a7d8a55 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:02.660610 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d7ea1-12b9-4732-9ba5-e4a80a7d8a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:02.728116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-595d7ea1-12b9-4732-9ba5-e4a80a7d8a55 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:02.729334 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-595d7ea1-12b9-4732-9ba5-e4a80a7d8a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:02.825983 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:02.828578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 660/1348] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:03.116306 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2aad03-8750-477a-8173-a2f8d09e7e5f None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:03.117962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-cf2aad03-8750-477a-8173-a2f8d09e7e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:03.248629 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2aad03-8750-477a-8173-a2f8d09e7e5f None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:03.310896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:03.340623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adda64f-a0d9-4a90-907d-ce7a0f94e7a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:03.341233 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adda64f-a0d9-4a90-907d-ce7a0f94e7a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:03.341520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adda64f-a0d9-4a90-907d-ce7a0f94e7a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:03.341790 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adda64f-a0d9-4a90-907d-ce7a0f94e7a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:03.343431 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-8adda64f-a0d9-4a90-907d-ce7a0f94e7a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:03.456478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adda64f-a0d9-4a90-907d-ce7a0f94e7a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:03.457912 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adda64f-a0d9-4a90-907d-ce7a0f94e7a1 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:03.475665 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 689/1349] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:08.350645 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde49788-38ba-4257-86e1-115a3df1ec54 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:08.351190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde49788-38ba-4257-86e1-115a3df1ec54 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:08.351190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde49788-38ba-4257-86e1-115a3df1ec54 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:08.352881 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 661/1350] 10.222.0.81 () {54 vars in 921 bytes} [Mon Sep 19 11:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:08.358039 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5a483-b6a4-4aa9-a7ea-66e402058db4 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:08.358478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5a483-b6a4-4aa9-a7ea-66e402058db4 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:08.358576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5a483-b6a4-4aa9-a7ea-66e402058db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:08.458145 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-f8c5a483-b6a4-4aa9-a7ea-66e402058db4 None None] MFA Rules not processed for user `6e7a18dc4a4d4d35a05f1aafbaaf9c31`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:08.459469 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f8c5a483-b6a4-4aa9-a7ea-66e402058db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:08.490864 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:08.638714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 690/1351] 10.222.0.81 () {58 vars in 1018 bytes} [Mon Sep 19 11:16:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:12.690752 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73726bfb-763b-4647-82f1-67335d96c607 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:12.691557 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73726bfb-763b-4647-82f1-67335d96c607 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:12.691557 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73726bfb-763b-4647-82f1-67335d96c607 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:12.693117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 662/1352] 10.222.0.81 () {54 vars in 921 bytes} [Mon Sep 19 11:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:12.698059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27146568-77c4-4386-9c21-daea294d9264 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:12.698529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27146568-77c4-4386-9c21-daea294d9264 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:12.698922 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27146568-77c4-4386-9c21-daea294d9264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:12.779581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-27146568-77c4-4386-9c21-daea294d9264 None None] MFA Rules not processed for user `6e7a18dc4a4d4d35a05f1aafbaaf9c31`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:12.780828 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-27146568-77c4-4386-9c21-daea294d9264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:12.857413 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:12.899920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 691/1353] 10.222.0.81 () {58 vars in 1018 bytes} [Mon Sep 19 11:16:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:14.503273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbdfde-1c20-4002-8d81-05b87ef92499 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:14.503859 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbdfde-1c20-4002-8d81-05b87ef92499 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:14.503956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbdfde-1c20-4002-8d81-05b87ef92499 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:14.505407 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 663/1354] 10.222.0.81 () {54 vars in 921 bytes} [Mon Sep 19 11:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:14.510721 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8783ce-5b83-4c5c-a4b3-b3b573af3886 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:14.511152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8783ce-5b83-4c5c-a4b3-b3b573af3886 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:14.511281 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8783ce-5b83-4c5c-a4b3-b3b573af3886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:14.589015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-cf8783ce-5b83-4c5c-a4b3-b3b573af3886 None None] MFA Rules not processed for user `6e7a18dc4a4d4d35a05f1aafbaaf9c31`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:14.590223 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-cf8783ce-5b83-4c5c-a4b3-b3b573af3886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:14.709901 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 692/1355] 10.222.0.81 () {58 vars in 1018 bytes} [Mon Sep 19 11:16:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:18.113919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccf60e-27b7-4304-90be-30cce0a30378 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:18.114454 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccf60e-27b7-4304-90be-30cce0a30378 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:18.114454 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccf60e-27b7-4304-90be-30cce0a30378 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:18.117609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 664/1356] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:16:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:18.124275 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bcec5-ca16-4fd2-945b-5d2e66d33b7c None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:18.124275 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bcec5-ca16-4fd2-945b-5d2e66d33b7c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:18.124829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bcec5-ca16-4fd2-945b-5d2e66d33b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:18.173188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:18.206614 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-589bcec5-ca16-4fd2-945b-5d2e66d33b7c None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:18.207737 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-589bcec5-ca16-4fd2-945b-5d2e66d33b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:18.314556 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 693/1357] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:18.622303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edd420-6d8b-42e1-a8c6-88f1c76389d0 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:18.622875 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edd420-6d8b-42e1-a8c6-88f1c76389d0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:18.623363 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edd420-6d8b-42e1-a8c6-88f1c76389d0 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:18.625346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 665/1358] 10.222.0.81 () {54 vars in 965 bytes} [Mon Sep 19 11:16:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:18.631855 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c160c-28b6-40e7-9803-ec0d66a3f2e6 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:18.632398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c160c-28b6-40e7-9803-ec0d66a3f2e6 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:18.632898 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c160c-28b6-40e7-9803-ec0d66a3f2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:18.734156 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-b70c160c-28b6-40e7-9803-ec0d66a3f2e6 None None] MFA Rules not processed for user `5abd70a18ccd407bafe3b61c8870986e`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:18.736485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b70c160c-28b6-40e7-9803-ec0d66a3f2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:18.790549 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:18.871806 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 694/1359] 10.222.0.81 () {58 vars in 1062 bytes} [Mon Sep 19 11:16:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:18.879496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4edd420-6d8b-42e1-a8c6-88f1c76389d0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:18.881153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d4edd420-6d8b-42e1-a8c6-88f1c76389d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:19.040801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4edd420-6d8b-42e1-a8c6-88f1c76389d0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:19.120801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:19.169964 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239fc24-ca99-48b5-828f-85d27995b827 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:19.170647 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239fc24-ca99-48b5-828f-85d27995b827 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:19.171039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239fc24-ca99-48b5-828f-85d27995b827 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:19.171421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239fc24-ca99-48b5-828f-85d27995b827 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:19.173157 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5239fc24-ca99-48b5-828f-85d27995b827 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:19.278488 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239fc24-ca99-48b5-828f-85d27995b827 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:19.280304 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239fc24-ca99-48b5-828f-85d27995b827 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:19.303271 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 666/1360] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:21.698755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3dddb1-3aca-4609-a4ba-e8f3256163a6 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:21.699383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3dddb1-3aca-4609-a4ba-e8f3256163a6 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:21.699383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3dddb1-3aca-4609-a4ba-e8f3256163a6 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:21.708372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd26f0-57c4-43a9-8416-dfcf268ed7db None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:21.708974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd26f0-57c4-43a9-8416-dfcf268ed7db None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:21.708974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd26f0-57c4-43a9-8416-dfcf268ed7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:21.711722 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 695/1361] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:16:21 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:21.793803 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d2cd26f0-57c4-43a9-8416-dfcf268ed7db None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:21.795059 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d2cd26f0-57c4-43a9-8416-dfcf268ed7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:21.835681 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:21.900283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 667/1362] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:22.190939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3dddb1-3aca-4609-a4ba-e8f3256163a6 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:22.192220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-bb3dddb1-3aca-4609-a4ba-e8f3256163a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:22.370756 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3dddb1-3aca-4609-a4ba-e8f3256163a6 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:22.381547 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:22.500719 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b48194-db2f-45dd-9b80-4ae222b78dcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:22.501373 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b48194-db2f-45dd-9b80-4ae222b78dcd service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:22.501661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b48194-db2f-45dd-9b80-4ae222b78dcd service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:22.502451 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b48194-db2f-45dd-9b80-4ae222b78dcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:22.504063 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a0b48194-db2f-45dd-9b80-4ae222b78dcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:22.594907 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b48194-db2f-45dd-9b80-4ae222b78dcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:22.597152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b48194-db2f-45dd-9b80-4ae222b78dcd service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:22.615607 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 696/1363] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:25.123480 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec61f4-e536-4c7c-ac8a-c90bed8fc43e None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:25.124091 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec61f4-e536-4c7c-ac8a-c90bed8fc43e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:25.124091 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec61f4-e536-4c7c-ac8a-c90bed8fc43e None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:25.125529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 668/1364] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:25.131089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5580d55-2b4a-467f-9ca8-f8cb1bf6fa5f None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:25.131514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5580d55-2b4a-467f-9ca8-f8cb1bf6fa5f None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:25.131700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5580d55-2b4a-467f-9ca8-f8cb1bf6fa5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:25.146917 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:25.200191 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-f5580d55-2b4a-467f-9ca8-f8cb1bf6fa5f None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:25.201300 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f5580d55-2b4a-467f-9ca8-f8cb1bf6fa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:25.305726 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 697/1365] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:16:25.603909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee2e8a-4733-4d61-a62c-a9fab4b5864e None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:25.604194 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee2e8a-4733-4d61-a62c-a9fab4b5864e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:25.604468 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee2e8a-4733-4d61-a62c-a9fab4b5864e None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:25.605902 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 669/1366] 10.222.0.81 () {54 vars in 965 bytes} [Mon Sep 19 11:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:25.611462 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff63a7-e88b-4c94-92a6-6f1bbb5f1cec None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:25.611840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff63a7-e88b-4c94-92a6-6f1bbb5f1cec None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:25.612197 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff63a7-e88b-4c94-92a6-6f1bbb5f1cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:25.694343 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-39ff63a7-e88b-4c94-92a6-6f1bbb5f1cec None None] MFA Rules not processed for user `5abd70a18ccd407bafe3b61c8870986e`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:25.695743 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-39ff63a7-e88b-4c94-92a6-6f1bbb5f1cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:25.731630 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:25.817416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 698/1367] 10.222.0.81 () {58 vars in 1062 bytes} [Mon Sep 19 11:16:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:25.825175 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee2e8a-4733-4d61-a62c-a9fab4b5864e None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:25.829662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7eee2e8a-4733-4d61-a62c-a9fab4b5864e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:25.980613 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee2e8a-4733-4d61-a62c-a9fab4b5864e None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:26.001571 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:26.103345 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68390d42-40dc-4f20-ae9a-8c7a5f552432 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:26.103984 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68390d42-40dc-4f20-ae9a-8c7a5f552432 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:26.104267 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68390d42-40dc-4f20-ae9a-8c7a5f552432 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:26.104540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68390d42-40dc-4f20-ae9a-8c7a5f552432 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:26.106390 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-68390d42-40dc-4f20-ae9a-8c7a5f552432 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:26.216083 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68390d42-40dc-4f20-ae9a-8c7a5f552432 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:26.218005 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68390d42-40dc-4f20-ae9a-8c7a5f552432 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:26.238958 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 670/1368] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:26.304416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05877d3-61a6-48d2-9178-b0aabf32c7e3 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:26.305270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05877d3-61a6-48d2-9178-b0aabf32c7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:26.305696 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05877d3-61a6-48d2-9178-b0aabf32c7e3 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:26.307155 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 699/1369] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:16:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:26.314083 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57b6d1-d647-4809-83a8-eabf55fdc696 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:26.314431 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57b6d1-d647-4809-83a8-eabf55fdc696 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:26.314905 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57b6d1-d647-4809-83a8-eabf55fdc696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:26.356699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:26.402021 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-dc57b6d1-d647-4809-83a8-eabf55fdc696 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:26.403473 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-dc57b6d1-d647-4809-83a8-eabf55fdc696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:26.532769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 671/1370] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:26.858982 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47505264-b0f3-40b9-a5aa-f0de651683e7 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:26.859276 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47505264-b0f3-40b9-a5aa-f0de651683e7 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:26.859556 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47505264-b0f3-40b9-a5aa-f0de651683e7 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:26.861589 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 700/1371] 10.222.0.81 () {54 vars in 967 bytes} [Mon Sep 19 11:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:26.866582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238a882-d4f6-45c8-b0c2-e4c8e7cc2795 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:26.866964 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238a882-d4f6-45c8-b0c2-e4c8e7cc2795 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:26.867311 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238a882-d4f6-45c8-b0c2-e4c8e7cc2795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:26.949777 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a238a882-d4f6-45c8-b0c2-e4c8e7cc2795 None None] MFA Rules not processed for user `f9f494df37254102b339df35a151e075`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:26.950989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a238a882-d4f6-45c8-b0c2-e4c8e7cc2795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:27.007996 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:27.042107 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 672/1372] 10.222.0.81 () {58 vars in 1064 bytes} [Mon Sep 19 11:16:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:27.050254 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47505264-b0f3-40b9-a5aa-f0de651683e7 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:27.052024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-47505264-b0f3-40b9-a5aa-f0de651683e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:27.166720 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47505264-b0f3-40b9-a5aa-f0de651683e7 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:27.212210 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:27.252963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da65f60-9512-4e59-90d3-d683a2ffd07c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:27.253639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da65f60-9512-4e59-90d3-d683a2ffd07c service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:27.253919 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da65f60-9512-4e59-90d3-d683a2ffd07c service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:27.254190 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da65f60-9512-4e59-90d3-d683a2ffd07c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:27.255979 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-7da65f60-9512-4e59-90d3-d683a2ffd07c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:27.389390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da65f60-9512-4e59-90d3-d683a2ffd07c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:27.392327 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da65f60-9512-4e59-90d3-d683a2ffd07c service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:27.413894 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 701/1373] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:28.492865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422a7b1-ecbb-41d7-acf7-5f98b98cb773 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:28.493406 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422a7b1-ecbb-41d7-acf7-5f98b98cb773 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:28.493406 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422a7b1-ecbb-41d7-acf7-5f98b98cb773 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:28.494705 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 673/1374] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:28.500166 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe78aee-756c-4037-89d8-c9e5245ae69e None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:28.500580 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe78aee-756c-4037-89d8-c9e5245ae69e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:28.500883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe78aee-756c-4037-89d8-c9e5245ae69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:28.572986 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-cbe78aee-756c-4037-89d8-c9e5245ae69e None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:28.574184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-cbe78aee-756c-4037-89d8-c9e5245ae69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:28.593991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:28.673969 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 702/1375] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:28.975121 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422a7b1-ecbb-41d7-acf7-5f98b98cb773 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:28.976714 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4422a7b1-ecbb-41d7-acf7-5f98b98cb773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:29.133721 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422a7b1-ecbb-41d7-acf7-5f98b98cb773 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:29.186064 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:29.250108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2d247-7993-42e5-97f5-bf42da9fa969 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:29.251593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2d247-7993-42e5-97f5-bf42da9fa969 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:29.251946 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2d247-7993-42e5-97f5-bf42da9fa969 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:29.252217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2d247-7993-42e5-97f5-bf42da9fa969 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:29.253833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b8a2d247-7993-42e5-97f5-bf42da9fa969 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:29.355598 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2d247-7993-42e5-97f5-bf42da9fa969 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:29.357234 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2d247-7993-42e5-97f5-bf42da9fa969 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:29.376974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 674/1376] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:31.630436 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0bdaf-71b6-4327-a799-52b93de97055 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:31.630961 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0bdaf-71b6-4327-a799-52b93de97055 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:31.630961 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0bdaf-71b6-4327-a799-52b93de97055 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:31.632278 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 703/1377] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:31.637736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9b461-24f4-40d0-81a2-5ab4bfb71978 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:31.638211 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9b461-24f4-40d0-81a2-5ab4bfb71978 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:31.638306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9b461-24f4-40d0-81a2-5ab4bfb71978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:31.712094 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-69a9b461-24f4-40d0-81a2-5ab4bfb71978 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:31.713303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-69a9b461-24f4-40d0-81a2-5ab4bfb71978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:31.721330 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:31.823410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 675/1378] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:32.113927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0bdaf-71b6-4327-a799-52b93de97055 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:32.115677 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-1fc0bdaf-71b6-4327-a799-52b93de97055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:32.267431 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0bdaf-71b6-4327-a799-52b93de97055 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:32.271614 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:32.394384 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ca3cd-ccd1-4019-9760-1fd45d286ecf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:32.394906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ca3cd-ccd1-4019-9760-1fd45d286ecf service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:32.395226 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ca3cd-ccd1-4019-9760-1fd45d286ecf service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:32.395727 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ca3cd-ccd1-4019-9760-1fd45d286ecf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:32.397757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-115ca3cd-ccd1-4019-9760-1fd45d286ecf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:32.494885 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ca3cd-ccd1-4019-9760-1fd45d286ecf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:32.496608 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ca3cd-ccd1-4019-9760-1fd45d286ecf service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:32.515195 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 704/1379] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:16:34.740795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c1d11-9fec-4d91-a4de-66774f537266 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:34.741248 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c1d11-9fec-4d91-a4de-66774f537266 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:34.741425 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c1d11-9fec-4d91-a4de-66774f537266 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:34.744085 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 676/1380] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:16:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:34.748149 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34ba2f-eebc-45f9-820a-758ba1a9d5ba None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:34.748536 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34ba2f-eebc-45f9-820a-758ba1a9d5ba None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:34.748707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34ba2f-eebc-45f9-820a-758ba1a9d5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:34.760029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:34.863568 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-7e34ba2f-eebc-45f9-820a-758ba1a9d5ba None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:34.867023 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-7e34ba2f-eebc-45f9-820a-758ba1a9d5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:34.992301 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 705/1381] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:35.285074 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c1d11-9fec-4d91-a4de-66774f537266 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:35.286696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-923c1d11-9fec-4d91-a4de-66774f537266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:35.448831 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:35.461236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c1d11-9fec-4d91-a4de-66774f537266 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:35.591252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4452f-2bc2-4494-9c2b-74657e935298 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:35.591849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4452f-2bc2-4494-9c2b-74657e935298 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:35.592146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4452f-2bc2-4494-9c2b-74657e935298 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:35.592383 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4452f-2bc2-4494-9c2b-74657e935298 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:35.594118 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-77c4452f-2bc2-4494-9c2b-74657e935298 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:35.697953 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4452f-2bc2-4494-9c2b-74657e935298 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:35.699579 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4452f-2bc2-4494-9c2b-74657e935298 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:35.709754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:35.719935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 677/1382] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:38.548802 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf9cf2-c966-4527-bf43-07de1e29b05c None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:38.549338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf9cf2-c966-4527-bf43-07de1e29b05c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:38.549338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf9cf2-c966-4527-bf43-07de1e29b05c None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:38.550592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 706/1383] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:16:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:38.557279 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c79d7-6a5a-4360-bba3-ab5e90fd6315 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:38.557698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c79d7-6a5a-4360-bba3-ab5e90fd6315 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:38.557826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c79d7-6a5a-4360-bba3-ab5e90fd6315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:38.628382 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c05c79d7-6a5a-4360-bba3-ab5e90fd6315 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:38.629612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c05c79d7-6a5a-4360-bba3-ab5e90fd6315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:38.730483 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 678/1384] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:39.022804 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf9cf2-c966-4527-bf43-07de1e29b05c None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:39.024800 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2fdf9cf2-c966-4527-bf43-07de1e29b05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:39.139340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:39.215712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf9cf2-c966-4527-bf43-07de1e29b05c None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:39.335349 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ece49-25c8-4fee-85c2-cf81b5d32545 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:39.336019 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ece49-25c8-4fee-85c2-cf81b5d32545 service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:39.336316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ece49-25c8-4fee-85c2-cf81b5d32545 service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:39.336630 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ece49-25c8-4fee-85c2-cf81b5d32545 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:39.338324 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-060ece49-25c8-4fee-85c2-cf81b5d32545 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:39.394891 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:39.444232 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ece49-25c8-4fee-85c2-cf81b5d32545 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:39.446256 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ece49-25c8-4fee-85c2-cf81b5d32545 service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:39.467878 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 707/1385] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:41.784482 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826af448-50b1-4ad1-ac8d-f4272bcaa3ab None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:41.785021 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826af448-50b1-4ad1-ac8d-f4272bcaa3ab None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:41.785021 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826af448-50b1-4ad1-ac8d-f4272bcaa3ab None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:41.786449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 679/1386] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:41.791880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277de90-247a-43ac-9d6b-a90306c6ecfc None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:41.792351 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277de90-247a-43ac-9d6b-a90306c6ecfc None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:41.792461 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277de90-247a-43ac-9d6b-a90306c6ecfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:41.858251 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-2277de90-247a-43ac-9d6b-a90306c6ecfc None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:41.860018 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2277de90-247a-43ac-9d6b-a90306c6ecfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:41.956244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 708/1387] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:42.257460 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826af448-50b1-4ad1-ac8d-f4272bcaa3ab None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:42.259110 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-826af448-50b1-4ad1-ac8d-f4272bcaa3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:42.340334 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:42.416216 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826af448-50b1-4ad1-ac8d-f4272bcaa3ab None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:42.536680 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3409cf7-3090-4b3a-9618-a4b3c48a5cc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:42.537200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3409cf7-3090-4b3a-9618-a4b3c48a5cc6 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:42.537475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3409cf7-3090-4b3a-9618-a4b3c48a5cc6 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:42.537741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3409cf7-3090-4b3a-9618-a4b3c48a5cc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:42.539366 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b3409cf7-3090-4b3a-9618-a4b3c48a5cc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:42.579398 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:42.631213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3409cf7-3090-4b3a-9618-a4b3c48a5cc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:42.632836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3409cf7-3090-4b3a-9618-a4b3c48a5cc6 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:42.654404 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 680/1388] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:43.016977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b98363-c292-4de5-84a3-89d877d40a44 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:43.017558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b98363-c292-4de5-84a3-89d877d40a44 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:43.017558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b98363-c292-4de5-84a3-89d877d40a44 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:43.018876 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 709/1389] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:43.024298 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3fe7a-e412-4e66-8b82-204e18c0d149 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:43.024722 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3fe7a-e412-4e66-8b82-204e18c0d149 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:43.025038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3fe7a-e412-4e66-8b82-204e18c0d149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:43.091585 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-40f3fe7a-e412-4e66-8b82-204e18c0d149 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:43.092817 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-40f3fe7a-e412-4e66-8b82-204e18c0d149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:43.190971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 681/1390] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:43.496903 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb4fc8-112e-406c-83c0-529e11331295 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:43.497189 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb4fc8-112e-406c-83c0-529e11331295 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:43.497464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb4fc8-112e-406c-83c0-529e11331295 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:43.498782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 710/1391] 10.222.0.81 () {54 vars in 962 bytes} [Mon Sep 19 11:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:43.504496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88435fff-db4c-49a5-aebd-7383e2848ee9 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:43.504783 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88435fff-db4c-49a5-aebd-7383e2848ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:43.505058 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88435fff-db4c-49a5-aebd-7383e2848ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:43.521313 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:43.587065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-88435fff-db4c-49a5-aebd-7383e2848ee9 None None] MFA Rules not processed for user `88670b0e1a0f460d8f74917c92db9b9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:43.588304 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-88435fff-db4c-49a5-aebd-7383e2848ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:43.662764 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 682/1392] 10.222.0.81 () {58 vars in 1059 bytes} [Mon Sep 19 11:16:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:16:43.672107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb4fc8-112e-406c-83c0-529e11331295 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:43.673605 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e0bb4fc8-112e-406c-83c0-529e11331295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:43.704981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:43.778007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb4fc8-112e-406c-83c0-529e11331295 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:43.850789 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb57bc7-26d7-46e2-9006-5192b65f7439 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88670b0e1a0f460d8f74917c92db9b9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:43.851395 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb57bc7-26d7-46e2-9006-5192b65f7439 service glance] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:43.851674 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb57bc7-26d7-46e2-9006-5192b65f7439 service glance] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:43.851981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb57bc7-26d7-46e2-9006-5192b65f7439 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:43.853561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-feb57bc7-26d7-46e2-9006-5192b65f7439 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:43.930766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:43.963208 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb57bc7-26d7-46e2-9006-5192b65f7439 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:43.964688 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb57bc7-26d7-46e2-9006-5192b65f7439 service glance] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:43.983359 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 711/1393] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:44.747552 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c37bc3-52c9-4bdd-adb3-9f24daeb1b82 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:44.748068 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c37bc3-52c9-4bdd-adb3-9f24daeb1b82 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:44.748189 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c37bc3-52c9-4bdd-adb3-9f24daeb1b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:44.822399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-b2c37bc3-52c9-4bdd-adb3-9f24daeb1b82 None None] MFA Rules not processed for user `06d279b488d148428b7a3c35a9786ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:44.823619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b2c37bc3-52c9-4bdd-adb3-9f24daeb1b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:44.878867 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:44.890632 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf55c9-0e73-4c32-8e35-2a5dcec6cadc None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:44.891206 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf55c9-0e73-4c32-8e35-2a5dcec6cadc None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:44.891206 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf55c9-0e73-4c32-8e35-2a5dcec6cadc None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:44.892538 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 712/1394] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:16:44.897609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c7459-138e-4498-9b19-07bbe0fac08f None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:44.897897 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c7459-138e-4498-9b19-07bbe0fac08f None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:44.898176 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c7459-138e-4498-9b19-07bbe0fac08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:44.913278 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 683/1395] 10.222.0.81 () {58 vars in 1009 bytes} [Mon Sep 19 11:16:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:16:44.957646 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41199ced-509a-41a1-b729-483934b3e53b None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:44.957932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41199ced-509a-41a1-b729-483934b3e53b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:44.958203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41199ced-509a-41a1-b729-483934b3e53b None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:44.959527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 684/1396] 10.222.0.81 () {54 vars in 943 bytes} [Mon Sep 19 11:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:16:44.965695 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52a590-e168-4d48-aa36-f5532aaa3102 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:44.965983 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52a590-e168-4d48-aa36-f5532aaa3102 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:44.966261 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52a590-e168-4d48-aa36-f5532aaa3102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:44.970606 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-3a1c7459-138e-4498-9b19-07bbe0fac08f None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:44.971924 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3a1c7459-138e-4498-9b19-07bbe0fac08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:45.049353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6c52a590-e168-4d48-aa36-f5532aaa3102 None None] MFA Rules not processed for user `1c8a9732807d4921a41a6a2d5019e2f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:45.050568 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6c52a590-e168-4d48-aa36-f5532aaa3102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:45.076251 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 713/1397] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:16:45.162222 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 685/1398] 10.222.0.81 () {58 vars in 1040 bytes} [Mon Sep 19 11:16:44 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:16:45.171679 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c7459-138e-4498-9b19-07bbe0fac08f None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:45.172900 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3a1c7459-138e-4498-9b19-07bbe0fac08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:45.212849 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:45.279020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c7459-138e-4498-9b19-07bbe0fac08f None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:45.370318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52a590-e168-4d48-aa36-f5532aaa3102 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:45.371585 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6c52a590-e168-4d48-aa36-f5532aaa3102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:45.389100 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a862d5-055c-4e65-8403-4493479e61bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8a9732807d4921a41a6a2d5019e2f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:45.389733 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a862d5-055c-4e65-8403-4493479e61bb service swift] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:45.390825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a862d5-055c-4e65-8403-4493479e61bb service swift] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:45.391033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a862d5-055c-4e65-8403-4493479e61bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:45.392732 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-05a862d5-055c-4e65-8403-4493479e61bb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:45.404469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:45.449757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:45.480857 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a862d5-055c-4e65-8403-4493479e61bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:45.481807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52a590-e168-4d48-aa36-f5532aaa3102 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:45.482364 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a862d5-055c-4e65-8403-4493479e61bb service swift] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:45.491964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 714/1399] 10.222.0.81 () {60 vars in 1384 bytes} [Mon Sep 19 11:16:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:16:45.623576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca5ec9-b470-457e-8d30-ec14c075f04c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:45.624245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca5ec9-b470-457e-8d30-ec14c075f04c service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:45.625085 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca5ec9-b470-457e-8d30-ec14c075f04c service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:45.625360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca5ec9-b470-457e-8d30-ec14c075f04c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:45.626961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-68ca5ec9-b470-457e-8d30-ec14c075f04c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:45.675904 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:45.727428 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca5ec9-b470-457e-8d30-ec14c075f04c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:45.729072 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca5ec9-b470-457e-8d30-ec14c075f04c service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:45.748466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 686/1400] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:48.002553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ca042-e5fb-47dd-b782-da2c6d05857e None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:48.003449 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ca042-e5fb-47dd-b782-da2c6d05857e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:48.003937 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ca042-e5fb-47dd-b782-da2c6d05857e None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:48.005557 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 715/1401] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:16:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:48.010963 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faa35e-8cc5-46a7-87af-1e80c66c2834 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:48.011397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faa35e-8cc5-46a7-87af-1e80c66c2834 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:48.011557 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faa35e-8cc5-46a7-87af-1e80c66c2834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:48.087631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d1faa35e-8cc5-46a7-87af-1e80c66c2834 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:48.088892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d1faa35e-8cc5-46a7-87af-1e80c66c2834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:48.193409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 687/1402] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:48.487451 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ca042-e5fb-47dd-b782-da2c6d05857e None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:48.490634 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-078ca042-e5fb-47dd-b782-da2c6d05857e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:48.651551 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ca042-e5fb-47dd-b782-da2c6d05857e None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:48.714202 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:48.785128 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153c1eb-a77e-4c74-a2b9-c2ea31b8d8a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:48.785725 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153c1eb-a77e-4c74-a2b9-c2ea31b8d8a5 service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:48.786003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153c1eb-a77e-4c74-a2b9-c2ea31b8d8a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:48.790264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153c1eb-a77e-4c74-a2b9-c2ea31b8d8a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:48.790264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e153c1eb-a77e-4c74-a2b9-c2ea31b8d8a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:48.884315 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153c1eb-a77e-4c74-a2b9-c2ea31b8d8a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:48.885891 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153c1eb-a77e-4c74-a2b9-c2ea31b8d8a5 service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:48.906325 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 716/1403] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:16:51.174276 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8690bd-e11f-456f-aacc-fc2b8d9ea9a2 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:51.175283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8690bd-e11f-456f-aacc-fc2b8d9ea9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:51.175895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8690bd-e11f-456f-aacc-fc2b8d9ea9a2 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:51.179791 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 688/1404] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:16:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:51.182910 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc8a59-b492-4a00-9bf1-d932b9869f1c None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:51.183347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc8a59-b492-4a00-9bf1-d932b9869f1c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:51.183455 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc8a59-b492-4a00-9bf1-d932b9869f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:51.265693 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-ecdc8a59-b492-4a00-9bf1-d932b9869f1c None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:51.266946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ecdc8a59-b492-4a00-9bf1-d932b9869f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:51.270390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:51.400001 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 717/1405] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:51.698668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8690bd-e11f-456f-aacc-fc2b8d9ea9a2 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:51.700233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8e8690bd-e11f-456f-aacc-fc2b8d9ea9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:51.725275 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:51.871202 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8690bd-e11f-456f-aacc-fc2b8d9ea9a2 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:51.976883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:52.004038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e3acb-ab16-4ed5-a1c9-56b73562c1ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:52.004647 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e3acb-ab16-4ed5-a1c9-56b73562c1ea service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:52.004925 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e3acb-ab16-4ed5-a1c9-56b73562c1ea service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:52.005199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e3acb-ab16-4ed5-a1c9-56b73562c1ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:52.006828 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-595e3acb-ab16-4ed5-a1c9-56b73562c1ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:52.106002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e3acb-ab16-4ed5-a1c9-56b73562c1ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:52.107624 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e3acb-ab16-4ed5-a1c9-56b73562c1ea service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:52.127143 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 689/1406] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:16:54.323726 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66972d-8eb4-4524-a747-aa9c92d1df5a None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:54.324326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66972d-8eb4-4524-a747-aa9c92d1df5a None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:54.324326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66972d-8eb4-4524-a747-aa9c92d1df5a None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:54.325600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 718/1407] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:16:54.331041 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd587a-e057-4dfe-9683-49a28f047d3d None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:54.331414 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd587a-e057-4dfe-9683-49a28f047d3d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:54.331501 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd587a-e057-4dfe-9683-49a28f047d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:54.404446 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-fddd587a-e057-4dfe-9683-49a28f047d3d None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:16:54.407564 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-fddd587a-e057-4dfe-9683-49a28f047d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:54.555282 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:54.630152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 690/1408] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:16:54.939902 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66972d-8eb4-4524-a747-aa9c92d1df5a None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:16:54.941689 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-9b66972d-8eb4-4524-a747-aa9c92d1df5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:55.091664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:55.105065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66972d-8eb4-4524-a747-aa9c92d1df5a None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:16:55.234206 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043229a-9c2a-4350-8362-402e527148d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:16:55.235305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043229a-9c2a-4350-8362-402e527148d8 service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:16:55.235582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043229a-9c2a-4350-8362-402e527148d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:16:55.235894 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043229a-9c2a-4350-8362-402e527148d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:16:55.237490 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b043229a-9c2a-4350-8362-402e527148d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:16:55.340624 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043229a-9c2a-4350-8362-402e527148d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:16:55.342354 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043229a-9c2a-4350-8362-402e527148d8 service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:16:55.353475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:16:55.365899 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 719/1409] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:16:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:04.628315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=19 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:05.363334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:07.866862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f888ccc-18ab-4e4c-a1b9-f6e1edef2b4c None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:07.867418 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f888ccc-18ab-4e4c-a1b9-f6e1edef2b4c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:07.867418 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f888ccc-18ab-4e4c-a1b9-f6e1edef2b4c None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:07.868773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 691/1410] 10.222.0.81 () {54 vars in 918 bytes} [Mon Sep 19 11:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:07.873785 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ac17a-ad16-437a-8a56-51a7ebcb4399 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:07.874202 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ac17a-ad16-437a-8a56-51a7ebcb4399 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:07.874333 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ac17a-ad16-437a-8a56-51a7ebcb4399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:07.948752 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-fd0ac17a-ad16-437a-8a56-51a7ebcb4399 None None] MFA Rules not processed for user `6e7a18dc4a4d4d35a05f1aafbaaf9c31`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:07.949881 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-fd0ac17a-ad16-437a-8a56-51a7ebcb4399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:08.059694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 720/1411] 10.222.0.81 () {58 vars in 1015 bytes} [Mon Sep 19 11:17:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:08.074431 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab7233-358e-49b6-ac61-f741bd39cced None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:08.074718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab7233-358e-49b6-ac61-f741bd39cced None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:08.074991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab7233-358e-49b6-ac61-f741bd39cced None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:08.076362 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 692/1412] 10.222.0.81 () {54 vars in 918 bytes} [Mon Sep 19 11:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:08.081385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae2fd8-59c5-4b5a-9f9a-8524b03d60dc None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:08.081673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae2fd8-59c5-4b5a-9f9a-8524b03d60dc None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:08.081948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae2fd8-59c5-4b5a-9f9a-8524b03d60dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:08.154393 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:08.157886 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-d8ae2fd8-59c5-4b5a-9f9a-8524b03d60dc None None] MFA Rules not processed for user `5abd70a18ccd407bafe3b61c8870986e`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:08.159952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-d8ae2fd8-59c5-4b5a-9f9a-8524b03d60dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:08.274845 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 721/1413] 10.222.0.81 () {58 vars in 1015 bytes} [Mon Sep 19 11:17:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:10.888744 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9faf0-732e-48b0-ae09-cb5bf6bf5204 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:10.889569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9faf0-732e-48b0-ae09-cb5bf6bf5204 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:10.889569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9faf0-732e-48b0-ae09-cb5bf6bf5204 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:10.890915 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 693/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:10.896791 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6bcf6-2c85-4936-b6d7-5c3d2acc3ed0 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:10.897232 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6bcf6-2c85-4936-b6d7-5c3d2acc3ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:10.897339 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6bcf6-2c85-4936-b6d7-5c3d2acc3ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:10.964891 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-3db6bcf6-2c85-4936-b6d7-5c3d2acc3ed0 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:10.966021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3db6bcf6-2c85-4936-b6d7-5c3d2acc3ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:11.020583 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:11.062423 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 722/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:11.352628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9faf0-732e-48b0-ae09-cb5bf6bf5204 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:11.354290 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ddd9faf0-732e-48b0-ae09-cb5bf6bf5204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:11.500056 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9faf0-732e-48b0-ae09-cb5bf6bf5204 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:11.572707 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:11.611722 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e8184-fddb-4793-bb8d-90c260478c7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:11.612348 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e8184-fddb-4793-bb8d-90c260478c7f service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:11.612619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e8184-fddb-4793-bb8d-90c260478c7f service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:11.612881 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e8184-fddb-4793-bb8d-90c260478c7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:11.614461 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8d1e8184-fddb-4793-bb8d-90c260478c7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:11.703576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e8184-fddb-4793-bb8d-90c260478c7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:11.705225 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e8184-fddb-4793-bb8d-90c260478c7f service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:11.722663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 694/1416] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:12.869763 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdd9e2-4972-4492-9b86-c57fb39337b0 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:12.870333 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdd9e2-4972-4492-9b86-c57fb39337b0 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:12.870333 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdd9e2-4972-4492-9b86-c57fb39337b0 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:12.871641 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 723/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:12.883643 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587c903-7e72-4495-b860-a9d2dd65c6cb None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:12.884421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587c903-7e72-4495-b860-a9d2dd65c6cb None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:12.884421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587c903-7e72-4495-b860-a9d2dd65c6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:12.961695 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e587c903-7e72-4495-b860-a9d2dd65c6cb None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:12.962833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e587c903-7e72-4495-b860-a9d2dd65c6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:12.989199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:13.058358 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 695/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:13.210709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdd9e2-4972-4492-9b86-c57fb39337b0 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:13.212320 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-0ffdd9e2-4972-4492-9b86-c57fb39337b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:13.357010 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdd9e2-4972-4492-9b86-c57fb39337b0 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:13.396928 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:13.472689 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bbb116-e195-488d-aecf-99c89eb63708 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:13.473208 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bbb116-e195-488d-aecf-99c89eb63708 service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:13.473489 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bbb116-e195-488d-aecf-99c89eb63708 service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:13.473762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bbb116-e195-488d-aecf-99c89eb63708 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:13.475357 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e0bbb116-e195-488d-aecf-99c89eb63708 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:13.562956 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bbb116-e195-488d-aecf-99c89eb63708 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:13.565080 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bbb116-e195-488d-aecf-99c89eb63708 service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:13.583473 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 724/1419] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:14.700184 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b7985-4d09-4483-bf02-adf4c29fc55f None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:14.700746 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b7985-4d09-4483-bf02-adf4c29fc55f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:14.700746 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b7985-4d09-4483-bf02-adf4c29fc55f None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:14.702196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 696/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:14.708570 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f929c26-e877-4709-925b-22fd8f2eb050 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:14.709033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f929c26-e877-4709-925b-22fd8f2eb050 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:14.709152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f929c26-e877-4709-925b-22fd8f2eb050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:14.762334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:14.784180 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-6f929c26-e877-4709-925b-22fd8f2eb050 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:14.785367 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-6f929c26-e877-4709-925b-22fd8f2eb050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:14.881073 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 725/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:15.036059 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b7985-4d09-4483-bf02-adf4c29fc55f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:15.037527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-588b7985-4d09-4483-bf02-adf4c29fc55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:15.182475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b7985-4d09-4483-bf02-adf4c29fc55f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:15.195134 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:15.294881 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be7e5e-aea4-4924-8472-c0a92f83c9e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:15.295408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be7e5e-aea4-4924-8472-c0a92f83c9e9 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:15.295687 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be7e5e-aea4-4924-8472-c0a92f83c9e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:15.296058 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be7e5e-aea4-4924-8472-c0a92f83c9e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:15.297665 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c3be7e5e-aea4-4924-8472-c0a92f83c9e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:15.394273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be7e5e-aea4-4924-8472-c0a92f83c9e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:15.396037 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be7e5e-aea4-4924-8472-c0a92f83c9e9 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:15.413962 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 697/1422] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:16.547330 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e3d40-cfff-4dd2-bd85-31ed631054bd None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:16.547930 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e3d40-cfff-4dd2-bd85-31ed631054bd None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:16.547930 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e3d40-cfff-4dd2-bd85-31ed631054bd None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:16.549757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 726/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:16.556135 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b07ea-4e3a-479f-88d9-ea37e3d8d615 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:16.556582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b07ea-4e3a-479f-88d9-ea37e3d8d615 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:16.556682 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b07ea-4e3a-479f-88d9-ea37e3d8d615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:16.576424 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:16.633283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-752b07ea-4e3a-479f-88d9-ea37e3d8d615 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:16.634501 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-752b07ea-4e3a-479f-88d9-ea37e3d8d615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:16.733109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 698/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:17:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:16.883694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e3d40-cfff-4dd2-bd85-31ed631054bd None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:16.885236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-cf8e3d40-cfff-4dd2-bd85-31ed631054bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:17.001943 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:17.037825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e3d40-cfff-4dd2-bd85-31ed631054bd None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:17.151172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522113c1-e68f-4b3d-9c25-2821004c4a45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:17.151692 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522113c1-e68f-4b3d-9c25-2821004c4a45 service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:17.151994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522113c1-e68f-4b3d-9c25-2821004c4a45 service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:17.152261 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522113c1-e68f-4b3d-9c25-2821004c4a45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:17.153851 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-522113c1-e68f-4b3d-9c25-2821004c4a45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:17.228699 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:17.242916 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522113c1-e68f-4b3d-9c25-2821004c4a45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:17.244520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522113c1-e68f-4b3d-9c25-2821004c4a45 service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:17.262042 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 727/1425] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:18.368731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7b633-fd40-48c3-8100-7e04afc15a32 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:18.369300 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7b633-fd40-48c3-8100-7e04afc15a32 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:18.369300 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7b633-fd40-48c3-8100-7e04afc15a32 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:18.370640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 699/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:18.376850 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefe0b6-0c15-4a3a-8a82-43422b64bb78 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:18.377394 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefe0b6-0c15-4a3a-8a82-43422b64bb78 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:18.378043 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefe0b6-0c15-4a3a-8a82-43422b64bb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:18.455881 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-9aefe0b6-0c15-4a3a-8a82-43422b64bb78 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:18.457104 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-9aefe0b6-0c15-4a3a-8a82-43422b64bb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:18.552268 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 728/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:18.700686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7b633-fd40-48c3-8100-7e04afc15a32 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:18.702207 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4ed7b633-fd40-48c3-8100-7e04afc15a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:18.788428 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:18.850280 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7b633-fd40-48c3-8100-7e04afc15a32 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:18.961795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c303f-84d1-40ea-86aa-4bae005a9e03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:18.962314 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c303f-84d1-40ea-86aa-4bae005a9e03 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:18.962591 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c303f-84d1-40ea-86aa-4bae005a9e03 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:18.962862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c303f-84d1-40ea-86aa-4bae005a9e03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:18.964484 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-947c303f-84d1-40ea-86aa-4bae005a9e03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:19.012265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:19.053663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c303f-84d1-40ea-86aa-4bae005a9e03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:19.055268 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c303f-84d1-40ea-86aa-4bae005a9e03 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:19.073408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 700/1428] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:20.126487 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867eeab-ee75-4d54-8bc1-5e3eeab7c99c None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:20.127113 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867eeab-ee75-4d54-8bc1-5e3eeab7c99c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:20.127238 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867eeab-ee75-4d54-8bc1-5e3eeab7c99c None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:20.128614 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 729/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:20.135089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f41c2c-c25d-43e6-86fa-2c701884c175 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:20.135608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f41c2c-c25d-43e6-86fa-2c701884c175 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:20.135739 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f41c2c-c25d-43e6-86fa-2c701884c175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:20.210965 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-97f41c2c-c25d-43e6-86fa-2c701884c175 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:20.212267 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-97f41c2c-c25d-43e6-86fa-2c701884c175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:20.308374 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 701/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:17:20.457613 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867eeab-ee75-4d54-8bc1-5e3eeab7c99c None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:20.459122 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2867eeab-ee75-4d54-8bc1-5e3eeab7c99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:20.506801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:20.603366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867eeab-ee75-4d54-8bc1-5e3eeab7c99c None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:20.717186 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301461c-b7a9-46b6-8e0a-a34e16b2cce8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:20.717787 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301461c-b7a9-46b6-8e0a-a34e16b2cce8 service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:20.718062 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301461c-b7a9-46b6-8e0a-a34e16b2cce8 service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:20.718329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301461c-b7a9-46b6-8e0a-a34e16b2cce8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:20.719954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a301461c-b7a9-46b6-8e0a-a34e16b2cce8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:20.726797 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:20.807888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301461c-b7a9-46b6-8e0a-a34e16b2cce8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:20.809485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301461c-b7a9-46b6-8e0a-a34e16b2cce8 service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:20.827800 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 730/1431] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:21.889848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530abcc-bf7b-4f65-8fe9-4d6f2e181da0 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:21.890430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530abcc-bf7b-4f65-8fe9-4d6f2e181da0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:21.890430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530abcc-bf7b-4f65-8fe9-4d6f2e181da0 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:21.891733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 702/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:21.898639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44427081-ff65-403d-a26a-5c08f64131db None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:21.899220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44427081-ff65-403d-a26a-5c08f64131db None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:21.899220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44427081-ff65-403d-a26a-5c08f64131db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:21.974537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-44427081-ff65-403d-a26a-5c08f64131db None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:21.975668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-44427081-ff65-403d-a26a-5c08f64131db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:22.041735 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:22.071245 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 731/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:22.219730 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530abcc-bf7b-4f65-8fe9-4d6f2e181da0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:22.221320 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e530abcc-bf7b-4f65-8fe9-4d6f2e181da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:22.238602 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:22.365613 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530abcc-bf7b-4f65-8fe9-4d6f2e181da0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:22.451438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:22.476767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c31d1-4ada-42b1-bf88-ba1bdb19a8ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:22.477359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c31d1-4ada-42b1-bf88-ba1bdb19a8ba service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:22.477633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c31d1-4ada-42b1-bf88-ba1bdb19a8ba service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:22.477906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c31d1-4ada-42b1-bf88-ba1bdb19a8ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:22.479480 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a39c31d1-4ada-42b1-bf88-ba1bdb19a8ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:22.567634 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c31d1-4ada-42b1-bf88-ba1bdb19a8ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:22.569445 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c31d1-4ada-42b1-bf88-ba1bdb19a8ba service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:22.587014 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 703/1434] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:23.644281 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0af37-6836-4517-b646-6f36434a0166 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:23.644862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0af37-6836-4517-b646-6f36434a0166 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:23.644862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0af37-6836-4517-b646-6f36434a0166 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:23.646230 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 732/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:17:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:23.652537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae926a5-b34e-49da-8bf3-c9de32d22365 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:23.652966 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae926a5-b34e-49da-8bf3-c9de32d22365 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:23.653082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae926a5-b34e-49da-8bf3-c9de32d22365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:23.731031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-4ae926a5-b34e-49da-8bf3-c9de32d22365 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:23.732276 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4ae926a5-b34e-49da-8bf3-c9de32d22365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:23.774017 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:23.827317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 704/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:23.980298 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0af37-6836-4517-b646-6f36434a0166 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:23.980298 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-01e0af37-6836-4517-b646-6f36434a0166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:24.128561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0af37-6836-4517-b646-6f36434a0166 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:24.181596 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:24.240755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcca8e-3147-4292-a252-812c3c4f55f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:24.241317 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcca8e-3147-4292-a252-812c3c4f55f7 service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:24.241596 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcca8e-3147-4292-a252-812c3c4f55f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:24.241869 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcca8e-3147-4292-a252-812c3c4f55f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:24.243479 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-04fcca8e-3147-4292-a252-812c3c4f55f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:24.331891 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fcca8e-3147-4292-a252-812c3c4f55f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:24.333484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fcca8e-3147-4292-a252-812c3c4f55f7 service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:24.350909 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 733/1437] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:25.397777 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df472d-9f84-4fd0-84ce-abfd28225723 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:25.398321 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df472d-9f84-4fd0-84ce-abfd28225723 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:25.398321 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df472d-9f84-4fd0-84ce-abfd28225723 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:25.399660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 705/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:25.405809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09febc-44ac-4d07-bb47-accf1eabda22 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:25.406241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09febc-44ac-4d07-bb47-accf1eabda22 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:25.406390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09febc-44ac-4d07-bb47-accf1eabda22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:25.484277 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-2d09febc-44ac-4d07-bb47-accf1eabda22 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:25.485606 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2d09febc-44ac-4d07-bb47-accf1eabda22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:25.488151 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:25.581555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 734/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:17:25.730923 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df472d-9f84-4fd0-84ce-abfd28225723 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:25.732558 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f3df472d-9f84-4fd0-84ce-abfd28225723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:25.876902 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df472d-9f84-4fd0-84ce-abfd28225723 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:25.903838 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:25.989916 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08c7b8-3e43-4104-acb2-6cec03ee540e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:25.990445 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08c7b8-3e43-4104-acb2-6cec03ee540e service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:25.990724 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08c7b8-3e43-4104-acb2-6cec03ee540e service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:25.990995 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08c7b8-3e43-4104-acb2-6cec03ee540e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:25.992615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1e08c7b8-3e43-4104-acb2-6cec03ee540e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:26.081098 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08c7b8-3e43-4104-acb2-6cec03ee540e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:26.083173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08c7b8-3e43-4104-acb2-6cec03ee540e service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:26.100878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 706/1440] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:28.178620 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020408b-eb13-481e-8db9-07d79718b624 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:28.179199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020408b-eb13-481e-8db9-07d79718b624 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:28.179199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020408b-eb13-481e-8db9-07d79718b624 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:28.180552 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 735/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:28.185893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51249d-2aee-4dba-9288-4abc6c74d4e0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:28.186364 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51249d-2aee-4dba-9288-4abc6c74d4e0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:28.186480 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51249d-2aee-4dba-9288-4abc6c74d4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:28.227918 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:28.252912 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-ce51249d-2aee-4dba-9288-4abc6c74d4e0 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:28.254637 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ce51249d-2aee-4dba-9288-4abc6c74d4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:28.467048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 707/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:28.748907 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020408b-eb13-481e-8db9-07d79718b624 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:28.750478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3020408b-eb13-481e-8db9-07d79718b624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:28.853025 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020408b-eb13-481e-8db9-07d79718b624 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:28.885029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:28.926516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d63f574-19ee-43ab-9aea-9f92f2e4db0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:28.927109 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63f574-19ee-43ab-9aea-9f92f2e4db0a service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:28.927378 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63f574-19ee-43ab-9aea-9f92f2e4db0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:28.927638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63f574-19ee-43ab-9aea-9f92f2e4db0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:28.929241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5d63f574-19ee-43ab-9aea-9f92f2e4db0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:29.033345 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63f574-19ee-43ab-9aea-9f92f2e4db0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:29.035194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63f574-19ee-43ab-9aea-9f92f2e4db0a service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:29.053168 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 736/1443] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:31.368265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfd77c-1c2d-450e-9e34-e993ac605529 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:31.368788 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfd77c-1c2d-450e-9e34-e993ac605529 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:31.368788 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfd77c-1c2d-450e-9e34-e993ac605529 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:31.370148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 708/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:31.375545 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a77c5-0244-412f-ae7f-9d0af8ed7e5c None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:31.376030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a77c5-0244-412f-ae7f-9d0af8ed7e5c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:31.376141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a77c5-0244-412f-ae7f-9d0af8ed7e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:31.441579 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-831a77c5-0244-412f-ae7f-9d0af8ed7e5c None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:31.442785 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-831a77c5-0244-412f-ae7f-9d0af8ed7e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:31.455464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:31.538804 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 737/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:31.826364 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfd77c-1c2d-450e-9e34-e993ac605529 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:31.827962 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-73cfd77c-1c2d-450e-9e34-e993ac605529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:31.929617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfd77c-1c2d-450e-9e34-e993ac605529 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:31.941433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:32.002851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90345864-d2f0-42c0-ac19-8ecfb2959cd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:32.003495 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90345864-d2f0-42c0-ac19-8ecfb2959cd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:32.003791 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90345864-d2f0-42c0-ac19-8ecfb2959cd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:32.004073 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90345864-d2f0-42c0-ac19-8ecfb2959cd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:32.005650 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-90345864-d2f0-42c0-ac19-8ecfb2959cd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:32.113864 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90345864-d2f0-42c0-ac19-8ecfb2959cd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:32.115344 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90345864-d2f0-42c0-ac19-8ecfb2959cd6 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:32.138679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 709/1446] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:34.579628 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d3581-b7f8-4b1c-8dc3-ec2124faeb43 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:34.580213 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d3581-b7f8-4b1c-8dc3-ec2124faeb43 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:34.580213 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d3581-b7f8-4b1c-8dc3-ec2124faeb43 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:34.581522 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 738/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:34.586704 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626c9b8-e52b-4fdf-8582-450788f09523 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:34.587151 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626c9b8-e52b-4fdf-8582-450788f09523 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:34.587289 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626c9b8-e52b-4fdf-8582-450788f09523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:34.635064 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:34.653668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-3626c9b8-e52b-4fdf-8582-450788f09523 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:34.654884 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3626c9b8-e52b-4fdf-8582-450788f09523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:34.751063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 710/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:17:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:35.030111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d3581-b7f8-4b1c-8dc3-ec2124faeb43 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:35.031636 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-6f8d3581-b7f8-4b1c-8dc3-ec2124faeb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:35.134668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d3581-b7f8-4b1c-8dc3-ec2124faeb43 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:35.176610 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:35.210515 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624ba58-3370-4265-903f-3f45f07cf90f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:35.211116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624ba58-3370-4265-903f-3f45f07cf90f service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:35.211395 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624ba58-3370-4265-903f-3f45f07cf90f service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:35.211667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624ba58-3370-4265-903f-3f45f07cf90f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:35.213294 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e624ba58-3370-4265-903f-3f45f07cf90f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:35.321067 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624ba58-3370-4265-903f-3f45f07cf90f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:35.322416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624ba58-3370-4265-903f-3f45f07cf90f service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:35.340958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 739/1449] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:37.591622 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e20a2-1cdc-4718-bae1-c48e751c5988 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:37.592211 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e20a2-1cdc-4718-bae1-c48e751c5988 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:37.592211 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e20a2-1cdc-4718-bae1-c48e751c5988 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:37.593518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 711/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:37.598906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e29a4-a93e-43d9-87ac-37e2e365639a None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:37.599354 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e29a4-a93e-43d9-87ac-37e2e365639a None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:37.599471 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e29a4-a93e-43d9-87ac-37e2e365639a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:37.664802 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-551e29a4-a93e-43d9-87ac-37e2e365639a None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:37.666017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-551e29a4-a93e-43d9-87ac-37e2e365639a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:37.687129 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:37.764042 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 740/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:38.047059 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e20a2-1cdc-4718-bae1-c48e751c5988 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:38.048649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bb5e20a2-1cdc-4718-bae1-c48e751c5988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:38.149049 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e20a2-1cdc-4718-bae1-c48e751c5988 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:38.169486 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:38.221115 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c4830-f743-4e22-bd3e-c35afc7b9327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:38.221705 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c4830-f743-4e22-bd3e-c35afc7b9327 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:38.221979 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c4830-f743-4e22-bd3e-c35afc7b9327 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:38.222245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c4830-f743-4e22-bd3e-c35afc7b9327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:38.224387 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2e6c4830-f743-4e22-bd3e-c35afc7b9327 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:38.330080 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c4830-f743-4e22-bd3e-c35afc7b9327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:38.331438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c4830-f743-4e22-bd3e-c35afc7b9327 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:38.349881 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 712/1452] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:40.668768 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ee625-9ad0-4a7c-ad46-ce0ca483a043 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:40.669345 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ee625-9ad0-4a7c-ad46-ce0ca483a043 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:40.669345 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ee625-9ad0-4a7c-ad46-ce0ca483a043 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:40.670709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 741/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:40.676293 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870fd99-0fc9-4e5e-a640-269445346aab None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:40.676693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870fd99-0fc9-4e5e-a640-269445346aab None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:40.676860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870fd99-0fc9-4e5e-a640-269445346aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:40.740675 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:40.743722 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e870fd99-0fc9-4e5e-a640-269445346aab None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:40.745631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e870fd99-0fc9-4e5e-a640-269445346aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:40.841556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 713/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:41.123755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ee625-9ad0-4a7c-ad46-ce0ca483a043 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:41.125318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-6c9ee625-9ad0-4a7c-ad46-ce0ca483a043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:41.227843 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ee625-9ad0-4a7c-ad46-ce0ca483a043 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:41.277989 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:41.302233 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15bd1b-def9-4a33-8dd1-9390b5a48f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:41.302825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15bd1b-def9-4a33-8dd1-9390b5a48f99 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:41.303119 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15bd1b-def9-4a33-8dd1-9390b5a48f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:41.303370 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15bd1b-def9-4a33-8dd1-9390b5a48f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:41.304975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-cf15bd1b-def9-4a33-8dd1-9390b5a48f99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:41.411732 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15bd1b-def9-4a33-8dd1-9390b5a48f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:41.413217 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15bd1b-def9-4a33-8dd1-9390b5a48f99 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:41.432062 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 742/1455] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:43.637581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5a287-d8ec-4d2f-86d5-58d7a9193f23 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:43.638160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5a287-d8ec-4d2f-86d5-58d7a9193f23 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:43.638160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5a287-d8ec-4d2f-86d5-58d7a9193f23 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:43.639468 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 714/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:43.644791 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc791d3-88a2-4c7f-811d-ff8facb412a3 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:43.645711 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc791d3-88a2-4c7f-811d-ff8facb412a3 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:43.645824 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc791d3-88a2-4c7f-811d-ff8facb412a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:43.711134 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-bbc791d3-88a2-4c7f-811d-ff8facb412a3 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:43.712360 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-bbc791d3-88a2-4c7f-811d-ff8facb412a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:43.740935 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:43.807342 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 743/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:44.094761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5a287-d8ec-4d2f-86d5-58d7a9193f23 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:44.096404 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-30a5a287-d8ec-4d2f-86d5-58d7a9193f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:44.199416 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5a287-d8ec-4d2f-86d5-58d7a9193f23 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:44.226992 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:44.270704 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bd08c-fa67-44a3-8f0d-0728b6d89e26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:44.271288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bd08c-fa67-44a3-8f0d-0728b6d89e26 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:44.271561 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bd08c-fa67-44a3-8f0d-0728b6d89e26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:44.271859 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bd08c-fa67-44a3-8f0d-0728b6d89e26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:44.273433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-831bd08c-fa67-44a3-8f0d-0728b6d89e26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:44.380332 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bd08c-fa67-44a3-8f0d-0728b6d89e26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:44.381772 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bd08c-fa67-44a3-8f0d-0728b6d89e26 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:44.399645 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 715/1458] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:46.601922 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a44be-b1c8-47ba-b10e-6d113ec7166f None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:46.602490 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a44be-b1c8-47ba-b10e-6d113ec7166f None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:46.602490 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a44be-b1c8-47ba-b10e-6d113ec7166f None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:46.603833 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 744/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:46.610511 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f499a-5c70-4cbd-8e3d-78c55e421423 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:46.610953 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f499a-5c70-4cbd-8e3d-78c55e421423 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:46.611061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f499a-5c70-4cbd-8e3d-78c55e421423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:46.679821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-7b2f499a-5c70-4cbd-8e3d-78c55e421423 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:46.681046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7b2f499a-5c70-4cbd-8e3d-78c55e421423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:46.689443 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:46.776146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 716/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:47.059275 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a44be-b1c8-47ba-b10e-6d113ec7166f None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:47.060826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-571a44be-b1c8-47ba-b10e-6d113ec7166f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:47.163395 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a44be-b1c8-47ba-b10e-6d113ec7166f None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:47.218794 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:47.236787 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e81777-c319-4e95-8f8b-5b15e0dbfa85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:47.237494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e81777-c319-4e95-8f8b-5b15e0dbfa85 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:47.237774 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e81777-c319-4e95-8f8b-5b15e0dbfa85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:47.238045 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e81777-c319-4e95-8f8b-5b15e0dbfa85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:47.239682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-d4e81777-c319-4e95-8f8b-5b15e0dbfa85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:47.472880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e81777-c319-4e95-8f8b-5b15e0dbfa85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:47.474341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e81777-c319-4e95-8f8b-5b15e0dbfa85 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:47.492389 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 745/1461] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:49.684008 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473e1e5-9d8e-4868-8cb1-45d87dec40d4 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:49.684433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473e1e5-9d8e-4868-8cb1-45d87dec40d4 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:49.684542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473e1e5-9d8e-4868-8cb1-45d87dec40d4 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:49.685893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 717/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:49.691251 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183090fb-a9da-4f30-a7eb-22ecd56e4dd7 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:49.691667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183090fb-a9da-4f30-a7eb-22ecd56e4dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:49.691829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183090fb-a9da-4f30-a7eb-22ecd56e4dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:49.757149 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-183090fb-a9da-4f30-a7eb-22ecd56e4dd7 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:49.758380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-183090fb-a9da-4f30-a7eb-22ecd56e4dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:49.798201 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:49.856411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 746/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:50.141215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473e1e5-9d8e-4868-8cb1-45d87dec40d4 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:50.142762 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4473e1e5-9d8e-4868-8cb1-45d87dec40d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:50.246144 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473e1e5-9d8e-4868-8cb1-45d87dec40d4 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:50.284413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:50.320834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a621d3d-1526-42c5-9349-c776a2ac200c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:50.321434 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a621d3d-1526-42c5-9349-c776a2ac200c service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:50.321971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a621d3d-1526-42c5-9349-c776a2ac200c service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:50.322237 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a621d3d-1526-42c5-9349-c776a2ac200c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:50.323850 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9a621d3d-1526-42c5-9349-c776a2ac200c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:50.429971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a621d3d-1526-42c5-9349-c776a2ac200c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:50.431412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a621d3d-1526-42c5-9349-c776a2ac200c service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:50.449843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 718/1464] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:52.586009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c863ec4-7e29-4c64-86a8-9a1eecdcebad None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:52.586640 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c863ec4-7e29-4c64-86a8-9a1eecdcebad None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:52.586640 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c863ec4-7e29-4c64-86a8-9a1eecdcebad None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:52.588200 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 747/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:52.594673 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2373b-2b2e-4993-98b8-180d572f9752 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:52.595181 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2373b-2b2e-4993-98b8-180d572f9752 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:52.595684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2373b-2b2e-4993-98b8-180d572f9752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:52.661385 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-27b2373b-2b2e-4993-98b8-180d572f9752 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:52.662583 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-27b2373b-2b2e-4993-98b8-180d572f9752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:52.682206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:52.760640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 719/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:53.048601 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c863ec4-7e29-4c64-86a8-9a1eecdcebad None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:53.050158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-9c863ec4-7e29-4c64-86a8-9a1eecdcebad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:53.152350 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c863ec4-7e29-4c64-86a8-9a1eecdcebad None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:53.220664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:53.228243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516719a1-ed55-423e-9569-d8af1028f9d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:53.228846 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516719a1-ed55-423e-9569-d8af1028f9d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:53.229124 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516719a1-ed55-423e-9569-d8af1028f9d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:53.229396 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516719a1-ed55-423e-9569-d8af1028f9d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:53.231022 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-516719a1-ed55-423e-9569-d8af1028f9d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:53.340837 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516719a1-ed55-423e-9569-d8af1028f9d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:53.342255 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516719a1-ed55-423e-9569-d8af1028f9d8 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:53.360877 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 748/1467] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:17:55.516616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8340dae-20a8-4e72-9a00-d9e3d21d072b None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:55.517203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8340dae-20a8-4e72-9a00-d9e3d21d072b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:55.517203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8340dae-20a8-4e72-9a00-d9e3d21d072b None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:55.518503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 720/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:17:55.524365 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f7be7-d417-4f6e-b5e6-85cf8e31a1cf None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:55.524820 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f7be7-d417-4f6e-b5e6-85cf8e31a1cf None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:55.524939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f7be7-d417-4f6e-b5e6-85cf8e31a1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:55.590343 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-529f7be7-d417-4f6e-b5e6-85cf8e31a1cf None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:17:55.591593 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-529f7be7-d417-4f6e-b5e6-85cf8e31a1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:55.638840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:55.688722 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 749/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:17:55.968888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8340dae-20a8-4e72-9a00-d9e3d21d072b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:17:55.970415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b8340dae-20a8-4e72-9a00-d9e3d21d072b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:56.072972 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8340dae-20a8-4e72-9a00-d9e3d21d072b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:17:56.123935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:17:56.149898 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67684f42-3246-48bf-9a30-82eeea43bffb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:17:56.150489 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67684f42-3246-48bf-9a30-82eeea43bffb service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:17:56.150780 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67684f42-3246-48bf-9a30-82eeea43bffb service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:17:56.151024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67684f42-3246-48bf-9a30-82eeea43bffb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:17:56.152635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-67684f42-3246-48bf-9a30-82eeea43bffb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:17:56.257420 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67684f42-3246-48bf-9a30-82eeea43bffb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:17:56.258768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67684f42-3246-48bf-9a30-82eeea43bffb service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:17:56.276619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 721/1470] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:17:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:18:05.687190 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=22 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:06.274400 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=66 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:16.203279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c71ebe-6590-4655-a188-1e60cd3bbbb6 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:16.203896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c71ebe-6590-4655-a188-1e60cd3bbbb6 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:16.203896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c71ebe-6590-4655-a188-1e60cd3bbbb6 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:16.205199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 750/1471] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:16.210920 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276d89d-7296-477d-a2ee-2693e6a831cc None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:16.211324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276d89d-7296-477d-a2ee-2693e6a831cc None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:16.211523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276d89d-7296-477d-a2ee-2693e6a831cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:16.277291 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-7276d89d-7296-477d-a2ee-2693e6a831cc None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:16.278525 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7276d89d-7296-477d-a2ee-2693e6a831cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:16.379477 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 722/1472] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:16.679938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee82a23-d0fb-4381-9dc4-4e6e5b2aceca None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:16.680227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee82a23-d0fb-4381-9dc4-4e6e5b2aceca None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:16.680505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee82a23-d0fb-4381-9dc4-4e6e5b2aceca None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:16.681823 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 751/1473] 10.222.0.81 () {54 vars in 962 bytes} [Mon Sep 19 11:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:16.686662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d50ff9-22f0-4239-9c61-6884089a17f1 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:16.686971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d50ff9-22f0-4239-9c61-6884089a17f1 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:16.687239 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d50ff9-22f0-4239-9c61-6884089a17f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:16.761600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-03d50ff9-22f0-4239-9c61-6884089a17f1 None None] MFA Rules not processed for user `88670b0e1a0f460d8f74917c92db9b9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:16.762797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-03d50ff9-22f0-4239-9c61-6884089a17f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:16.783715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:16.840553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 723/1474] 10.222.0.81 () {58 vars in 1059 bytes} [Mon Sep 19 11:18:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:16.848188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee82a23-d0fb-4381-9dc4-4e6e5b2aceca None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:16.849673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-dee82a23-d0fb-4381-9dc4-4e6e5b2aceca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:16.951341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee82a23-d0fb-4381-9dc4-4e6e5b2aceca None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:17.023931 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b899c5a-d575-44f3-86e7-cf7c0502c8dd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88670b0e1a0f460d8f74917c92db9b9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:17.024534 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b899c5a-d575-44f3-86e7-cf7c0502c8dd service glance] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:17.024812 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b899c5a-d575-44f3-86e7-cf7c0502c8dd service glance] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:17.025081 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b899c5a-d575-44f3-86e7-cf7c0502c8dd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:17.026670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-4b899c5a-d575-44f3-86e7-cf7c0502c8dd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:17.080667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:17.136214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b899c5a-d575-44f3-86e7-cf7c0502c8dd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:17.137641 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b899c5a-d575-44f3-86e7-cf7c0502c8dd service glance] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:17.155540 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 752/1475] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:18:19.424764 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cd378-d444-48e4-ab81-468eec8c59b2 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:19.425379 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cd378-d444-48e4-ab81-468eec8c59b2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:19.425379 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cd378-d444-48e4-ab81-468eec8c59b2 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:19.426560 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 724/1476] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:19.432571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcc7a1-2d7a-4a00-b67a-d35e94097678 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:19.433009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcc7a1-2d7a-4a00-b67a-d35e94097678 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:19.433125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcc7a1-2d7a-4a00-b67a-d35e94097678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:19.500932 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-26dcc7a1-2d7a-4a00-b67a-d35e94097678 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:19.502626 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-26dcc7a1-2d7a-4a00-b67a-d35e94097678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:19.605758 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 753/1477] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:19.883210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736cd378-d444-48e4-ab81-468eec8c59b2 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:19.884842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-736cd378-d444-48e4-ab81-468eec8c59b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:20.008685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736cd378-d444-48e4-ab81-468eec8c59b2 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:20.056229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:20.102849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f772c34-21f5-4de9-b11d-0e35dc344cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:20.103558 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f772c34-21f5-4de9-b11d-0e35dc344cfe admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:20.103865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f772c34-21f5-4de9-b11d-0e35dc344cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:20.104135 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f772c34-21f5-4de9-b11d-0e35dc344cfe admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:20.104771 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f772c34-21f5-4de9-b11d-0e35dc344cfe admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:20.106509 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: WARNING py.warnings [None req-4f772c34-21f5-4de9-b11d-0e35dc344cfe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:18:20.106509 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: warnings.warn(msg) Sep 19 11:18:20.106509 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: Sep 19 11:18:20.107634 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f772c34-21f5-4de9-b11d-0e35dc344cfe admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:20.126328 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 725/1478] 10.222.0.81 () {56 vars in 1108 bytes} [Mon Sep 19 11:18:19 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:18:22.259149 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f57cfb-85b5-47b7-a1e6-c02fe24d7da9 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:22.259781 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f57cfb-85b5-47b7-a1e6-c02fe24d7da9 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:22.259781 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f57cfb-85b5-47b7-a1e6-c02fe24d7da9 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:22.261097 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 754/1479] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:22.266394 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e1c13-0ccb-4fa5-b5d0-420a745eca52 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:22.266809 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e1c13-0ccb-4fa5-b5d0-420a745eca52 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:22.266935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e1c13-0ccb-4fa5-b5d0-420a745eca52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:22.332683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9f6e1c13-0ccb-4fa5-b5d0-420a745eca52 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:22.333931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9f6e1c13-0ccb-4fa5-b5d0-420a745eca52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:22.430056 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 726/1480] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:22.724181 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f57cfb-85b5-47b7-a1e6-c02fe24d7da9 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:22.725947 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f6f57cfb-85b5-47b7-a1e6-c02fe24d7da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:22.735708 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:22.869751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f57cfb-85b5-47b7-a1e6-c02fe24d7da9 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:22.952659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:22.984432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed422d0a-47f6-435f-a421-2220bdc3fb2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:22.985027 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed422d0a-47f6-435f-a421-2220bdc3fb2c service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:22.985304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed422d0a-47f6-435f-a421-2220bdc3fb2c service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:22.985566 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed422d0a-47f6-435f-a421-2220bdc3fb2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:22.987138 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ed422d0a-47f6-435f-a421-2220bdc3fb2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:23.074760 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed422d0a-47f6-435f-a421-2220bdc3fb2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:23.076360 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed422d0a-47f6-435f-a421-2220bdc3fb2c service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:23.093663 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 755/1481] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:18:25.405736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5e4cc-7e4e-4762-91d4-6a8f87e104c5 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:25.406271 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5e4cc-7e4e-4762-91d4-6a8f87e104c5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:25.406271 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5e4cc-7e4e-4762-91d4-6a8f87e104c5 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:25.407540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 727/1482] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:25.413954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa84fbc-ff08-4801-a946-443f5f41cb49 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:25.414411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa84fbc-ff08-4801-a946-443f5f41cb49 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:25.414531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa84fbc-ff08-4801-a946-443f5f41cb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:25.479909 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-ffa84fbc-ff08-4801-a946-443f5f41cb49 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:25.481113 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ffa84fbc-ff08-4801-a946-443f5f41cb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:25.526195 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:25.576757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 756/1483] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:25.870264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5e4cc-7e4e-4762-91d4-6a8f87e104c5 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:25.871847 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-65f5e4cc-7e4e-4762-91d4-6a8f87e104c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:25.895516 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:25.976821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5e4cc-7e4e-4762-91d4-6a8f87e104c5 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:26.048515 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a030c-50ab-4578-a43a-72a1b14d5f7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:26.049118 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a030c-50ab-4578-a43a-72a1b14d5f7c service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:26.049393 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a030c-50ab-4578-a43a-72a1b14d5f7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:26.049662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a030c-50ab-4578-a43a-72a1b14d5f7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:26.051244 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f55a030c-50ab-4578-a43a-72a1b14d5f7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:26.124727 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:26.158567 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a030c-50ab-4578-a43a-72a1b14d5f7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:26.160032 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a030c-50ab-4578-a43a-72a1b14d5f7c service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:26.178184 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 728/1484] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:18:28.387164 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e674b-82f0-4453-909d-b63cf04e2dfa None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:28.387750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e674b-82f0-4453-909d-b63cf04e2dfa None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:28.387750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e674b-82f0-4453-909d-b63cf04e2dfa None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:28.389085 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 757/1485] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:28.395163 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87ea60-09ce-4b6d-8a58-0a86f2e99726 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:28.395584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87ea60-09ce-4b6d-8a58-0a86f2e99726 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:28.395738 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87ea60-09ce-4b6d-8a58-0a86f2e99726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:28.463021 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-2d87ea60-09ce-4b6d-8a58-0a86f2e99726 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:28.464273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2d87ea60-09ce-4b6d-8a58-0a86f2e99726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:28.559048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 729/1486] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:28.852595 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e674b-82f0-4453-909d-b63cf04e2dfa None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:28.854154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f82e674b-82f0-4453-909d-b63cf04e2dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:28.957096 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e674b-82f0-4453-909d-b63cf04e2dfa None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:29.027975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:29.029423 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7294ffe-a635-4946-b5c8-3f85998d102f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:29.030070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7294ffe-a635-4946-b5c8-3f85998d102f service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:29.030399 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7294ffe-a635-4946-b5c8-3f85998d102f service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:29.030910 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7294ffe-a635-4946-b5c8-3f85998d102f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:29.032973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b7294ffe-a635-4946-b5c8-3f85998d102f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:29.142553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7294ffe-a635-4946-b5c8-3f85998d102f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:29.144020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7294ffe-a635-4946-b5c8-3f85998d102f service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:29.162171 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 758/1487] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:18:31.408571 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af63f0f-e715-478a-929f-760c4eecde81 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:31.409145 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af63f0f-e715-478a-929f-760c4eecde81 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:31.409145 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af63f0f-e715-478a-929f-760c4eecde81 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:31.410451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 730/1488] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:31.416393 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d067760-2e05-412e-ab8b-5293b000d2ba None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:31.416857 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d067760-2e05-412e-ab8b-5293b000d2ba None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:31.416968 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d067760-2e05-412e-ab8b-5293b000d2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:31.482695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-8d067760-2e05-412e-ab8b-5293b000d2ba None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:31.483940 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-8d067760-2e05-412e-ab8b-5293b000d2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:31.535913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:31.580867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 759/1489] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:31.878154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af63f0f-e715-478a-929f-760c4eecde81 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:31.879736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3af63f0f-e715-478a-929f-760c4eecde81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:31.916404 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:31.985154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af63f0f-e715-478a-929f-760c4eecde81 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:32.058177 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1a30d-ac75-415c-baa9-9d8dd311ed55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:32.058780 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1a30d-ac75-415c-baa9-9d8dd311ed55 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:32.059054 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1a30d-ac75-415c-baa9-9d8dd311ed55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:32.059322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1a30d-ac75-415c-baa9-9d8dd311ed55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:32.060946 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a7f1a30d-ac75-415c-baa9-9d8dd311ed55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:32.141963 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:32.174424 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1a30d-ac75-415c-baa9-9d8dd311ed55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:32.175897 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1a30d-ac75-415c-baa9-9d8dd311ed55 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:32.193873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 731/1490] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:18:34.570308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7254979-736c-4878-a00c-abd012a8a83c None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:34.570889 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7254979-736c-4878-a00c-abd012a8a83c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:34.570889 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7254979-736c-4878-a00c-abd012a8a83c None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:34.572266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 760/1491] 10.222.0.81 () {54 vars in 927 bytes} [Mon Sep 19 11:18:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:18:34.577893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8df450-662d-42eb-862f-8851e9a2ae96 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:34.578332 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8df450-662d-42eb-862f-8851e9a2ae96 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:34.578446 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8df450-662d-42eb-862f-8851e9a2ae96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:34.643316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6e8df450-662d-42eb-862f-8851e9a2ae96 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:18:34.644566 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6e8df450-662d-42eb-862f-8851e9a2ae96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:34.740118 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 732/1492] 10.222.0.81 () {58 vars in 1024 bytes} [Mon Sep 19 11:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:18:35.032372 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7254979-736c-4878-a00c-abd012a8a83c None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:18:35.033924 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a7254979-736c-4878-a00c-abd012a8a83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:35.136888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7254979-736c-4878-a00c-abd012a8a83c None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:18:35.223667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97625bd1-b6b6-41ac-b8a3-771e4b9b590b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:18:35.224439 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97625bd1-b6b6-41ac-b8a3-771e4b9b590b service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:18:35.224771 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97625bd1-b6b6-41ac-b8a3-771e4b9b590b service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:18:35.225099 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97625bd1-b6b6-41ac-b8a3-771e4b9b590b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:18:35.227511 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-97625bd1-b6b6-41ac-b8a3-771e4b9b590b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:18:35.237498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:35.355459 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97625bd1-b6b6-41ac-b8a3-771e4b9b590b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:18:35.357204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97625bd1-b6b6-41ac-b8a3-771e4b9b590b service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:18:35.378726 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 761/1493] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:18:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:18:44.738473 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=82 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:18:45.376111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=50 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:48.019571 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8eb74-8ca2-490d-ab15-6d80d2bc8428 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:48.020180 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8eb74-8ca2-490d-ab15-6d80d2bc8428 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:48.020180 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8eb74-8ca2-490d-ab15-6d80d2bc8428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:48.094562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-b4b8eb74-8ca2-490d-ab15-6d80d2bc8428 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:19:48.095790 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b4b8eb74-8ca2-490d-ab15-6d80d2bc8428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:48.191601 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 733/1494] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:19:48.199923 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504489f-f675-453d-9171-f406beb6d1ae None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:48.200416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504489f-f675-453d-9171-f406beb6d1ae None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:48.200530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504489f-f675-453d-9171-f406beb6d1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:48.277240 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-0504489f-f675-453d-9171-f406beb6d1ae None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:19:48.278379 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-0504489f-f675-453d-9171-f406beb6d1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:48.378890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 762/1495] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:19:48.383287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8eb74-8ca2-490d-ab15-6d80d2bc8428 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:48.384547 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b4b8eb74-8ca2-490d-ab15-6d80d2bc8428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:48.398817 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8eb74-8ca2-490d-ab15-6d80d2bc8428 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:48.472871 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:48.508089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e71949-850f-4f02-a26e-cc3f3d017e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:48.508751 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e71949-850f-4f02-a26e-cc3f3d017e2f admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:48.509031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e71949-850f-4f02-a26e-cc3f3d017e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:48.509858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e71949-850f-4f02-a26e-cc3f3d017e2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:48.510508 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e71949-850f-4f02-a26e-cc3f3d017e2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:19:48.510808 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e71949-850f-4f02-a26e-cc3f3d017e2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:48.511853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: WARNING py.warnings [None req-c3e71949-850f-4f02-a26e-cc3f3d017e2f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:19:48.511853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: warnings.warn(msg) Sep 19 11:19:48.511853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: Sep 19 11:19:48.512794 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e71949-850f-4f02-a26e-cc3f3d017e2f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:48.525689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 734/1496] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:19:48.533723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504489f-f675-453d-9171-f406beb6d1ae None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:48.545809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504489f-f675-453d-9171-f406beb6d1ae None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:48.607121 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:48.640624 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370e1df-5465-4726-8d1a-57a5c7d765bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:48.641221 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370e1df-5465-4726-8d1a-57a5c7d765bf admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:48.641496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370e1df-5465-4726-8d1a-57a5c7d765bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:48.641764 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370e1df-5465-4726-8d1a-57a5c7d765bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:48.642392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370e1df-5465-4726-8d1a-57a5c7d765bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:48.644146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: WARNING py.warnings [None req-1370e1df-5465-4726-8d1a-57a5c7d765bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:19:48.644146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: warnings.warn(msg) Sep 19 11:19:48.644146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: Sep 19 11:19:48.645326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370e1df-5465-4726-8d1a-57a5c7d765bf admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:48.700682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 763/1497] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:19:48 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:19:48.705061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e71949-850f-4f02-a26e-cc3f3d017e2f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:48.706605 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c3e71949-850f-4f02-a26e-cc3f3d017e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:48.740487 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e71949-850f-4f02-a26e-cc3f3d017e2f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:48.832627 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3689d9-0170-4c16-98ba-027f3ab6f856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:48.833148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3689d9-0170-4c16-98ba-027f3ab6f856 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:48.833424 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3689d9-0170-4c16-98ba-027f3ab6f856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:48.833691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3689d9-0170-4c16-98ba-027f3ab6f856 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:48.834319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3689d9-0170-4c16-98ba-027f3ab6f856 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:48.835335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: WARNING py.warnings [None req-7a3689d9-0170-4c16-98ba-027f3ab6f856 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:19:48.835335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: warnings.warn(msg) Sep 19 11:19:48.835335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: Sep 19 11:19:48.836558 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3689d9-0170-4c16-98ba-027f3ab6f856 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:48.878577 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 735/1498] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:19:48 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:19:48.882675 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370e1df-5465-4726-8d1a-57a5c7d765bf admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:48.884221 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-1370e1df-5465-4726-8d1a-57a5c7d765bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:48.945676 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370e1df-5465-4726-8d1a-57a5c7d765bf admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:49.029228 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:49.040407 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892af39-d4aa-48e4-8dd4-33aaf828b849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:49.041559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892af39-d4aa-48e4-8dd4-33aaf828b849 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:49.041819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892af39-d4aa-48e4-8dd4-33aaf828b849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:49.042084 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892af39-d4aa-48e4-8dd4-33aaf828b849 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:49.042715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a892af39-d4aa-48e4-8dd4-33aaf828b849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:49.043717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: WARNING py.warnings [None req-a892af39-d4aa-48e4-8dd4-33aaf828b849 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:19:49.043717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: warnings.warn(msg) Sep 19 11:19:49.043717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: Sep 19 11:19:49.044610 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a892af39-d4aa-48e4-8dd4-33aaf828b849 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:49.056816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 764/1499] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:19:49.061023 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3689d9-0170-4c16-98ba-027f3ab6f856 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:49.062550 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7a3689d9-0170-4c16-98ba-027f3ab6f856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:49.073942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3689d9-0170-4c16-98ba-027f3ab6f856 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:49.106182 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:49.177926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6de7ab-2409-447a-be18-db7baeb517e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:49.178529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6de7ab-2409-447a-be18-db7baeb517e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:49.178809 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6de7ab-2409-447a-be18-db7baeb517e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:49.179108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6de7ab-2409-447a-be18-db7baeb517e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:49.179749 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6de7ab-2409-447a-be18-db7baeb517e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:49.180793 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: WARNING py.warnings [None req-0b6de7ab-2409-447a-be18-db7baeb517e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:19:49.180793 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: warnings.warn(msg) Sep 19 11:19:49.180793 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: Sep 19 11:19:49.181718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6de7ab-2409-447a-be18-db7baeb517e2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:49.193961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 736/1500] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:19:49.198791 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892af39-d4aa-48e4-8dd4-33aaf828b849 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:49.200311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a892af39-d4aa-48e4-8dd4-33aaf828b849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:49.233088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892af39-d4aa-48e4-8dd4-33aaf828b849 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:49.325986 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0e54e-7861-46e7-8222-e8fe8bd26943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:49.326878 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0e54e-7861-46e7-8222-e8fe8bd26943 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:49.327155 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0e54e-7861-46e7-8222-e8fe8bd26943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:49.327427 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0e54e-7861-46e7-8222-e8fe8bd26943 admin admin] PATH_INFO: `/v3/projects/b3dbabb15dd845fd8f0064e8d3354898/users/0b7b2f3501c84c3682ab78ed3120649f/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:49.350797 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0e54e-7861-46e7-8222-e8fe8bd26943 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3dbabb15dd845fd8f0064e8d3354898, user_id=0b7b2f3501c84c3682ab78ed3120649f, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:49.351887 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: WARNING py.warnings [None req-aab0e54e-7861-46e7-8222-e8fe8bd26943 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:19:49.351887 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: warnings.warn(msg) Sep 19 11:19:49.351887 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: Sep 19 11:19:49.353858 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0e54e-7861-46e7-8222-e8fe8bd26943 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:49.377498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 765/1501] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:19:49 2022] PUT /identity/v3/projects/b3dbabb15dd845fd8f0064e8d3354898/users/0b7b2f3501c84c3682ab78ed3120649f/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:19:49.390218 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc19a2-258d-4bea-9695-0afa2511c32e None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:49.390501 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc19a2-258d-4bea-9695-0afa2511c32e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:49.390773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc19a2-258d-4bea-9695-0afa2511c32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:49.447278 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-abfc19a2-258d-4bea-9695-0afa2511c32e None None] MFA Rules not processed for user `0b7b2f3501c84c3682ab78ed3120649f`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:19:49.448549 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-abfc19a2-258d-4bea-9695-0afa2511c32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:49.528631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 737/1502] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:19:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:19:49.541832 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897cc4b-3002-49cf-bc3e-4f4041e2164c None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:49.542124 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897cc4b-3002-49cf-bc3e-4f4041e2164c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:49.542402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897cc4b-3002-49cf-bc3e-4f4041e2164c None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:49.543750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 766/1503] 10.222.0.81 () {54 vars in 967 bytes} [Mon Sep 19 11:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:19:49.548527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c4ec5-ee09-46f1-8084-6762b4401d46 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:49.548810 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c4ec5-ee09-46f1-8084-6762b4401d46 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:49.549081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c4ec5-ee09-46f1-8084-6762b4401d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:49.580496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:49.623712 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e51c4ec5-ee09-46f1-8084-6762b4401d46 None None] MFA Rules not processed for user `f9f494df37254102b339df35a151e075`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:19:49.624945 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e51c4ec5-ee09-46f1-8084-6762b4401d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:49.698880 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 738/1504] 10.222.0.81 () {58 vars in 1064 bytes} [Mon Sep 19 11:19:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:19:49.706778 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897cc4b-3002-49cf-bc3e-4f4041e2164c None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:49.708325 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a897cc4b-3002-49cf-bc3e-4f4041e2164c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:49.756597 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:49.812873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897cc4b-3002-49cf-bc3e-4f4041e2164c None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:49.885664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda3faf-f8dd-4db8-82ca-90f2b72e950a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:49.886258 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda3faf-f8dd-4db8-82ca-90f2b72e950a service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:49.886531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda3faf-f8dd-4db8-82ca-90f2b72e950a service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:49.886797 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda3faf-f8dd-4db8-82ca-90f2b72e950a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:49.888400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5fda3faf-f8dd-4db8-82ca-90f2b72e950a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:49.985991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda3faf-f8dd-4db8-82ca-90f2b72e950a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:49.987318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda3faf-f8dd-4db8-82ca-90f2b72e950a service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:49.990733 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:50.005188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 767/1505] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:19:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:19:50.049964 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163500f-ff2f-492d-b42f-d5de671471fe None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:50.050254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163500f-ff2f-492d-b42f-d5de671471fe None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:50.050530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163500f-ff2f-492d-b42f-d5de671471fe None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:50.051837 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 739/1506] 10.222.0.81 () {52 vars in 830 bytes} [Mon Sep 19 11:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:19:50.055451 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda3faf-f8dd-4db8-82ca-90f2b72e950a service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:50.056735 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5fda3faf-f8dd-4db8-82ca-90f2b72e950a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:50.157657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda3faf-f8dd-4db8-82ca-90f2b72e950a service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:50.248533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf10608-b7dd-41b5-9d6d-802c67f19e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:50.249174 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf10608-b7dd-41b5-9d6d-802c67f19e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:50.249346 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf10608-b7dd-41b5-9d6d-802c67f19e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:50.249616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf10608-b7dd-41b5-9d6d-802c67f19e6a admin admin] PATH_INFO: `/v3/users/0b7b2f3501c84c3682ab78ed3120649f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:50.262618 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf10608-b7dd-41b5-9d6d-802c67f19e6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b7b2f3501c84c3682ab78ed3120649f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:50.263634 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: WARNING py.warnings [None req-faf10608-b7dd-41b5-9d6d-802c67f19e6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:19:50.263634 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: warnings.warn(msg) Sep 19 11:19:50.263634 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: Sep 19 11:19:50.264875 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf10608-b7dd-41b5-9d6d-802c67f19e6a admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:50.274299 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:50.331413 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-faf10608-b7dd-41b5-9d6d-802c67f19e6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b7b2f3501c84c3682ab78ed3120649f'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:19:50.331965 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-faf10608-b7dd-41b5-9d6d-802c67f19e6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b7b2f3501c84c3682ab78ed3120649f'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:19:50.332383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-faf10608-b7dd-41b5-9d6d-802c67f19e6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b7b2f3501c84c3682ab78ed3120649f'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:19:50.346107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-faf10608-b7dd-41b5-9d6d-802c67f19e6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b7b2f3501c84c3682ab78ed3120649f'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:19:50.368316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-faf10608-b7dd-41b5-9d6d-802c67f19e6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b7b2f3501c84c3682ab78ed3120649f'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:19:50.383472 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 768/1507] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:19:50 2022] DELETE /identity/v3/users/0b7b2f3501c84c3682ab78ed3120649f => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:19:50.388418 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8819b24-0bfb-4831-8e53-e77c57c7d844 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:50.388944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8819b24-0bfb-4831-8e53-e77c57c7d844 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:50.389106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8819b24-0bfb-4831-8e53-e77c57c7d844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:50.445549 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f8819b24-0bfb-4831-8e53-e77c57c7d844 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:19:50.446752 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f8819b24-0bfb-4831-8e53-e77c57c7d844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:50.520644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:50.533126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 740/1508] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:19:50.542730 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf10608-b7dd-41b5-9d6d-802c67f19e6a admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:50.543990 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-faf10608-b7dd-41b5-9d6d-802c67f19e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:50.645284 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf10608-b7dd-41b5-9d6d-802c67f19e6a admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:50.700531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:19:50.721552 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3329573-00be-4af4-b514-853a88e9c460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:50.722154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3329573-00be-4af4-b514-853a88e9c460 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:50.722430 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3329573-00be-4af4-b514-853a88e9c460 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:50.722698 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3329573-00be-4af4-b514-853a88e9c460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:50.724328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a3329573-00be-4af4-b514-853a88e9c460 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:50.830947 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3329573-00be-4af4-b514-853a88e9c460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:50.832316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3329573-00be-4af4-b514-853a88e9c460 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:50.851259 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 769/1509] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:19:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:19:51.362158 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8819b24-0bfb-4831-8e53-e77c57c7d844 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:19:51.364019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f8819b24-0bfb-4831-8e53-e77c57c7d844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:19:51.394913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8819b24-0bfb-4831-8e53-e77c57c7d844 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:19:51.497914 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb78c3-066a-4ee7-ae44-bea8406221e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:19:51.498540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb78c3-066a-4ee7-ae44-bea8406221e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:19:51.498814 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb78c3-066a-4ee7-ae44-bea8406221e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:19:51.499080 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb78c3-066a-4ee7-ae44-bea8406221e9 admin admin] PATH_INFO: `/v3/projects/b3dbabb15dd845fd8f0064e8d3354898` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:19:51.508831 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb78c3-066a-4ee7-ae44-bea8406221e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3dbabb15dd845fd8f0064e8d3354898)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:19:51.510364 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: WARNING py.warnings [None req-a2fb78c3-066a-4ee7-ae44-bea8406221e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:19:51.510364 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: warnings.warn(msg) Sep 19 11:19:51.510364 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: Sep 19 11:19:51.511548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb78c3-066a-4ee7-ae44-bea8406221e9 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:19:51.569544 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a2fb78c3-066a-4ee7-ae44-bea8406221e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3dbabb15dd845fd8f0064e8d3354898', 'request_id': 'req-a2fb78c3-066a-4ee7-ae44-bea8406221e9'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:19:51.583641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a2fb78c3-066a-4ee7-ae44-bea8406221e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3dbabb15dd845fd8f0064e8d3354898', 'request_id': 'req-a2fb78c3-066a-4ee7-ae44-bea8406221e9'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:19:51.590209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a2fb78c3-066a-4ee7-ae44-bea8406221e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3dbabb15dd845fd8f0064e8d3354898', 'request_id': 'req-a2fb78c3-066a-4ee7-ae44-bea8406221e9'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:19:51.615326 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a2fb78c3-066a-4ee7-ae44-bea8406221e9 admin admin] The token cache is being invalidate because project b3dbabb15dd845fd8f0064e8d3354898 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:19:51.615618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a2fb78c3-066a-4ee7-ae44-bea8406221e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:19:51.616083 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a2fb78c3-066a-4ee7-ae44-bea8406221e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:19:51.617510 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 741/1510] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:19:51 2022] DELETE /identity/v3/projects/b3dbabb15dd845fd8f0064e8d3354898 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:20:00.848118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=62 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:20:01.613348 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:21:25.414065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079302b8-a2a8-416f-9475-30c3db37d682 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:25.414688 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079302b8-a2a8-416f-9475-30c3db37d682 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:25.414688 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079302b8-a2a8-416f-9475-30c3db37d682 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:25.416690 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 770/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:21:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:21:25.421628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0243d1-d34e-493b-82a6-30e164d8fda7 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:25.422075 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0243d1-d34e-493b-82a6-30e164d8fda7 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:25.422173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0243d1-d34e-493b-82a6-30e164d8fda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:25.501955 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-4d0243d1-d34e-493b-82a6-30e164d8fda7 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:21:25.503093 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4d0243d1-d34e-493b-82a6-30e164d8fda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:25.605380 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 742/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:21:25.764144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079302b8-a2a8-416f-9475-30c3db37d682 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:21:25.766271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-079302b8-a2a8-416f-9475-30c3db37d682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:25.888897 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079302b8-a2a8-416f-9475-30c3db37d682 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:21:25.894416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6288e000-c0ce-4a31-887a-2f8cce9e5ddb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:21:25.895035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288e000-c0ce-4a31-887a-2f8cce9e5ddb service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:25.895298 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288e000-c0ce-4a31-887a-2f8cce9e5ddb service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:25.895571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6288e000-c0ce-4a31-887a-2f8cce9e5ddb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:25.904183 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6288e000-c0ce-4a31-887a-2f8cce9e5ddb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:21:25.905762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6288e000-c0ce-4a31-887a-2f8cce9e5ddb service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:21:25.915199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 771/1513] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:21:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:21:26.938798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3acdb-dc50-44ba-8398-26dc05734231 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:26.939399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3acdb-dc50-44ba-8398-26dc05734231 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:26.939399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3acdb-dc50-44ba-8398-26dc05734231 None None] PATH_INFO: `/v3` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:26.940667 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 743/1514] 10.222.0.72 () {54 vars in 925 bytes} [Mon Sep 19 11:21:26 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:21:26.946521 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb9f13-20c3-44fe-bf2d-4d85b26dd5eb None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:26.947000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb9f13-20c3-44fe-bf2d-4d85b26dd5eb None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:26.947119 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb9f13-20c3-44fe-bf2d-4d85b26dd5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:26.998799 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b2627-8fca-4243-af81-7380eeff1b7e None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:26.999087 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b2627-8fca-4243-af81-7380eeff1b7e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:26.999358 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b2627-8fca-4243-af81-7380eeff1b7e None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:27.000711 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 744/1515] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 11:21:27.006634 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec686e-d58a-40ac-a3ad-70924d0028db None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:27.006918 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec686e-d58a-40ac-a3ad-70924d0028db None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:27.007190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec686e-d58a-40ac-a3ad-70924d0028db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:27.008399 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-9adb9f13-20c3-44fe-bf2d-4d85b26dd5eb None None] MFA Rules not processed for user `6e7a18dc4a4d4d35a05f1aafbaaf9c31`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:21:27.009686 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-9adb9f13-20c3-44fe-bf2d-4d85b26dd5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:27.026963 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6bec686e-d58a-40ac-a3ad-70924d0028db None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:21:27.028159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6bec686e-d58a-40ac-a3ad-70924d0028db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:27.035856 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 745/1516] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:21:27.077636 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:21:27.083359 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 772/1517] 10.222.0.72 () {58 vars in 1015 bytes} [Mon Sep 19 11:21:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:21:27.186949 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec686e-d58a-40ac-a3ad-70924d0028db None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:21:27.199945 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec686e-d58a-40ac-a3ad-70924d0028db None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:21:27.205545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95a205-8e72-4d0c-9f05-7979fb7dff46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:21:27.206148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95a205-8e72-4d0c-9f05-7979fb7dff46 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:27.206431 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95a205-8e72-4d0c-9f05-7979fb7dff46 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:27.206702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95a205-8e72-4d0c-9f05-7979fb7dff46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:27.216004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a95a205-8e72-4d0c-9f05-7979fb7dff46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:21:27.217643 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a95a205-8e72-4d0c-9f05-7979fb7dff46 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:21:27.227971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 746/1518] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:21:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:21:30.673008 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef83ef-d599-4ce9-8fa3-c823b635fe25 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:30.673409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef83ef-d599-4ce9-8fa3-c823b635fe25 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:30.673614 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef83ef-d599-4ce9-8fa3-c823b635fe25 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:30.674992 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 773/1519] 10.222.0.81 () {54 vars in 961 bytes} [Mon Sep 19 11:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:21:30.680462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb7d93-1e28-4d42-8391-2929104bf808 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:30.680919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb7d93-1e28-4d42-8391-2929104bf808 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:30.681036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb7d93-1e28-4d42-8391-2929104bf808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:30.701502 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-00eb7d93-1e28-4d42-8391-2929104bf808 None None] MFA Rules not processed for user `6e7a18dc4a4d4d35a05f1aafbaaf9c31`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:21:30.702687 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-00eb7d93-1e28-4d42-8391-2929104bf808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:30.711530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 747/1520] 10.222.0.81 () {58 vars in 1058 bytes} [Mon Sep 19 11:21:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:21:30.719134 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef83ef-d599-4ce9-8fa3-c823b635fe25 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:21:30.732241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef83ef-d599-4ce9-8fa3-c823b635fe25 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:21:30.737215 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bd404-73f5-4323-a479-b0a55ae9ea49 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e7a18dc4a4d4d35a05f1aafbaaf9c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:21:30.737745 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bd404-73f5-4323-a479-b0a55ae9ea49 service placement] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:30.738020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bd404-73f5-4323-a479-b0a55ae9ea49 service placement] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:30.738286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bd404-73f5-4323-a479-b0a55ae9ea49 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:30.746853 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bd404-73f5-4323-a479-b0a55ae9ea49 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:21:30.748473 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bd404-73f5-4323-a479-b0a55ae9ea49 service placement] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:21:30.758121 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 774/1521] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:21:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:21:38.670622 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156e7b3-a8c4-4282-9ce7-e3e0a006abae None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:38.671163 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156e7b3-a8c4-4282-9ce7-e3e0a006abae None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:38.671283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156e7b3-a8c4-4282-9ce7-e3e0a006abae None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:38.672684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 748/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:21:38.679142 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d9515-88dd-42ca-bb08-2d6f91d264d6 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:38.679579 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d9515-88dd-42ca-bb08-2d6f91d264d6 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:38.679684 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d9515-88dd-42ca-bb08-2d6f91d264d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:38.699565 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-d32d9515-88dd-42ca-bb08-2d6f91d264d6 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:21:38.700796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-d32d9515-88dd-42ca-bb08-2d6f91d264d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:38.708880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 775/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:21:38.866561 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156e7b3-a8c4-4282-9ce7-e3e0a006abae None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:21:38.882736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156e7b3-a8c4-4282-9ce7-e3e0a006abae None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:21:38.888714 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10447009-ee54-40e2-b1f8-080a9b3d9849 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:21:38.889324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10447009-ee54-40e2-b1f8-080a9b3d9849 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:38.889592 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10447009-ee54-40e2-b1f8-080a9b3d9849 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:38.889867 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10447009-ee54-40e2-b1f8-080a9b3d9849 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:38.897805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10447009-ee54-40e2-b1f8-080a9b3d9849 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:21:38.899380 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10447009-ee54-40e2-b1f8-080a9b3d9849 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:21:38.908680 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 749/1524] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:21:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:21:39.975473 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828cced-0bd4-4d99-89d8-ef49fe36023a None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:39.976052 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828cced-0bd4-4d99-89d8-ef49fe36023a None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:39.976052 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828cced-0bd4-4d99-89d8-ef49fe36023a None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:39.977301 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 776/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 11:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:21:39.983554 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eab2b7-f290-4e98-95a3-e10a59bc3e5f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:39.984041 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eab2b7-f290-4e98-95a3-e10a59bc3e5f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:39.984147 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eab2b7-f290-4e98-95a3-e10a59bc3e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:40.004861 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c7eab2b7-f290-4e98-95a3-e10a59bc3e5f None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:21:40.006065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c7eab2b7-f290-4e98-95a3-e10a59bc3e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:40.014328 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 750/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 11:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:21:40.166080 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828cced-0bd4-4d99-89d8-ef49fe36023a None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:21:40.180043 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828cced-0bd4-4d99-89d8-ef49fe36023a None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:21:40.185388 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47f0ca-79b4-45f0-81b6-d41466eb6b66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:21:40.185963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47f0ca-79b4-45f0-81b6-d41466eb6b66 service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:40.186236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47f0ca-79b4-45f0-81b6-d41466eb6b66 service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:40.186501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47f0ca-79b4-45f0-81b6-d41466eb6b66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:40.194294 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47f0ca-79b4-45f0-81b6-d41466eb6b66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:21:40.195887 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47f0ca-79b4-45f0-81b6-d41466eb6b66 service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:21:40.205867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 777/1527] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:21:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:21:44.466926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1d792-cd82-4dcb-896c-19f1611305f1 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:44.467558 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1d792-cd82-4dcb-896c-19f1611305f1 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:44.467558 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1d792-cd82-4dcb-896c-19f1611305f1 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:44.468840 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 751/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:21:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:21:44.473926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848ad95-4eff-4b4e-969b-0ba43e3a1c3f None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:44.474371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848ad95-4eff-4b4e-969b-0ba43e3a1c3f None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:44.474485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848ad95-4eff-4b4e-969b-0ba43e3a1c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:44.494732 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-4848ad95-4eff-4b4e-969b-0ba43e3a1c3f None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:21:44.495957 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-4848ad95-4eff-4b4e-969b-0ba43e3a1c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:44.503802 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 778/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:21:44.794651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1d792-cd82-4dcb-896c-19f1611305f1 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:21:44.796342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1ad1d792-cd82-4dcb-896c-19f1611305f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:44.823315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:21:44.877129 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1d792-cd82-4dcb-896c-19f1611305f1 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:21:44.880673 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7339d74-2e84-47fb-a34f-d1e33504ded9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:21:44.881267 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7339d74-2e84-47fb-a34f-d1e33504ded9 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:44.881544 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7339d74-2e84-47fb-a34f-d1e33504ded9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:44.881815 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7339d74-2e84-47fb-a34f-d1e33504ded9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:44.891276 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7339d74-2e84-47fb-a34f-d1e33504ded9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:21:44.892759 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7339d74-2e84-47fb-a34f-d1e33504ded9 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:21:44.902221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 752/1530] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:21:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:21:47.056988 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e369e08-cd3c-41e8-9a0d-f992d7b4d285 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:47.057553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e369e08-cd3c-41e8-9a0d-f992d7b4d285 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:47.057553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e369e08-cd3c-41e8-9a0d-f992d7b4d285 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:47.058886 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 779/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:21:47.064282 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4e875-627f-407e-97f6-e36b45fb8a01 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:47.064723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4e875-627f-407e-97f6-e36b45fb8a01 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:47.064839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4e875-627f-407e-97f6-e36b45fb8a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:47.084627 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a2d4e875-627f-407e-97f6-e36b45fb8a01 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:21:47.085837 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a2d4e875-627f-407e-97f6-e36b45fb8a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:47.094360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 753/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:21:47.382866 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e369e08-cd3c-41e8-9a0d-f992d7b4d285 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:21:47.394235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e369e08-cd3c-41e8-9a0d-f992d7b4d285 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:21:47.398046 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09f8ef-e66a-48d0-bdf5-b38c9fba2ea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:21:47.398635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09f8ef-e66a-48d0-bdf5-b38c9fba2ea5 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:47.398912 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09f8ef-e66a-48d0-bdf5-b38c9fba2ea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:47.399183 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09f8ef-e66a-48d0-bdf5-b38c9fba2ea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:47.408487 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09f8ef-e66a-48d0-bdf5-b38c9fba2ea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:21:47.409902 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09f8ef-e66a-48d0-bdf5-b38c9fba2ea5 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:21:47.419234 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 780/1533] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:21:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:21:51.269081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb20062-a506-43e1-bd04-4eca21624108 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:51.269507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb20062-a506-43e1-bd04-4eca21624108 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:51.269657 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb20062-a506-43e1-bd04-4eca21624108 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:51.270999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 754/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:21:51.276941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d62277-ff4a-49b8-81c9-19e366bbe541 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:51.277403 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d62277-ff4a-49b8-81c9-19e366bbe541 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:51.277507 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d62277-ff4a-49b8-81c9-19e366bbe541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:51.297016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-23d62277-ff4a-49b8-81c9-19e366bbe541 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:21:51.298219 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-23d62277-ff4a-49b8-81c9-19e366bbe541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:51.305945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 781/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:21:51.615053 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb20062-a506-43e1-bd04-4eca21624108 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:21:51.616776 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6eb20062-a506-43e1-bd04-4eca21624108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:51.695272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb20062-a506-43e1-bd04-4eca21624108 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:21:51.699442 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7f6e1-9dad-45ac-840e-66218a8bba15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:21:51.700074 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7f6e1-9dad-45ac-840e-66218a8bba15 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:51.700356 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7f6e1-9dad-45ac-840e-66218a8bba15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:51.700631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7f6e1-9dad-45ac-840e-66218a8bba15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:51.709701 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa7f6e1-9dad-45ac-840e-66218a8bba15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:21:51.711126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa7f6e1-9dad-45ac-840e-66218a8bba15 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:21:51.720501 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 755/1536] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:21:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:21:53.857959 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636b793-0308-41a7-90ae-f3c588901308 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:53.858519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636b793-0308-41a7-90ae-f3c588901308 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:53.858519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636b793-0308-41a7-90ae-f3c588901308 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:53.859915 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 782/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 11:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:21:53.865174 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f6c1a-5b46-41ab-9103-1b8e120111a1 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:53.865615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f6c1a-5b46-41ab-9103-1b8e120111a1 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:53.865745 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f6c1a-5b46-41ab-9103-1b8e120111a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:53.886128 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-031f6c1a-5b46-41ab-9103-1b8e120111a1 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:21:53.887335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-031f6c1a-5b46-41ab-9103-1b8e120111a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:53.895322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 756/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 11:21:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:21:54.194270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636b793-0308-41a7-90ae-f3c588901308 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:21:54.195944 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-1636b793-0308-41a7-90ae-f3c588901308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:21:54.209432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636b793-0308-41a7-90ae-f3c588901308 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:21:54.212907 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80827e11-ae2c-401d-8696-e6fc85badccb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:21:54.214050 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80827e11-ae2c-401d-8696-e6fc85badccb service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:21:54.214323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80827e11-ae2c-401d-8696-e6fc85badccb service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:21:54.214586 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80827e11-ae2c-401d-8696-e6fc85badccb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:21:54.223202 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80827e11-ae2c-401d-8696-e6fc85badccb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:21:54.224635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80827e11-ae2c-401d-8696-e6fc85badccb service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:21:54.233777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 783/1539] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:22:03.880715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=66 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:22:04.229709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=34 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:19.113064 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e0286-10f2-4678-8f63-9e73e97b9116 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.114781 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e0286-10f2-4678-8f63-9e73e97b9116 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.114781 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e0286-10f2-4678-8f63-9e73e97b9116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.155492 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f22e0286-10f2-4678-8f63-9e73e97b9116 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:19.156800 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f22e0286-10f2-4678-8f63-9e73e97b9116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:19.173427 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 757/1540] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:23:19.183044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5ebc5-63a2-4daa-a43e-358079631fc5 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.183644 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5ebc5-63a2-4daa-a43e-358079631fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.183644 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5ebc5-63a2-4daa-a43e-358079631fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.215214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-8ce5ebc5-63a2-4daa-a43e-358079631fc5 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:19.216500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-8ce5ebc5-63a2-4daa-a43e-358079631fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:19.231107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 784/1541] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:19.235927 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e0286-10f2-4678-8f63-9e73e97b9116 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:19.247414 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42943a02-f459-488d-8fe1-964d8b2ad554 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.248059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42943a02-f459-488d-8fe1-964d8b2ad554 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.248348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42943a02-f459-488d-8fe1-964d8b2ad554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.249171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e0286-10f2-4678-8f63-9e73e97b9116 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:19.257502 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d168b-fb33-4113-86d7-1859f2fdba9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:19.258116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d168b-fb33-4113-86d7-1859f2fdba9f admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.258398 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d168b-fb33-4113-86d7-1859f2fdba9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.258669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d168b-fb33-4113-86d7-1859f2fdba9f admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.259322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400d168b-fb33-4113-86d7-1859f2fdba9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:23:19.259620 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400d168b-fb33-4113-86d7-1859f2fdba9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:19.261552 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400d168b-fb33-4113-86d7-1859f2fdba9f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:19.276477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-42943a02-f459-488d-8fe1-964d8b2ad554 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:19.277709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-42943a02-f459-488d-8fe1-964d8b2ad554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:19.287972 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 758/1542] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:23:19.289331 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689b855-7443-469a-9fac-0d3377c6341e None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.289686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689b855-7443-469a-9fac-0d3377c6341e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.289969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689b855-7443-469a-9fac-0d3377c6341e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.292881 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 785/1543] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:23:19.293686 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42943a02-f459-488d-8fe1-964d8b2ad554 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:19.308203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42943a02-f459-488d-8fe1-964d8b2ad554 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:19.315883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-1689b855-7443-469a-9fac-0d3377c6341e None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:19.317101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1689b855-7443-469a-9fac-0d3377c6341e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:19.319512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73e690-325c-496c-b1b5-45474c0f8cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:19.320158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73e690-325c-496c-b1b5-45474c0f8cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.320439 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73e690-325c-496c-b1b5-45474c0f8cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.320713 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73e690-325c-496c-b1b5-45474c0f8cc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.321480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73e690-325c-496c-b1b5-45474c0f8cc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:19.323444 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73e690-325c-496c-b1b5-45474c0f8cc2 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:19.333817 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 759/1544] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:19.335581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37387479-5289-42cd-a937-06b5f96edfa9 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.335898 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37387479-5289-42cd-a937-06b5f96edfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.336175 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37387479-5289-42cd-a937-06b5f96edfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.367388 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-37387479-5289-42cd-a937-06b5f96edfa9 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:19.368551 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-37387479-5289-42cd-a937-06b5f96edfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:19.384977 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 760/1545] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:19.390103 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d197e0-32d2-4eae-aaf3-d86a63c384ba None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.391141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d197e0-32d2-4eae-aaf3-d86a63c384ba None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.391467 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d197e0-32d2-4eae-aaf3-d86a63c384ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.396998 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 786/1546] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/projects => generated 403 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:19.397862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73e690-325c-496c-b1b5-45474c0f8cc2 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:19.421255 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-60d197e0-32d2-4eae-aaf3-d86a63c384ba None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:19.425788 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-60d197e0-32d2-4eae-aaf3-d86a63c384ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:19.494046 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73e690-325c-496c-b1b5-45474c0f8cc2 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:19.500587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9efec0-25ed-42f4-a607-49306de661bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:19.501197 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9efec0-25ed-42f4-a607-49306de661bc admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.501476 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9efec0-25ed-42f4-a607-49306de661bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.501745 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9efec0-25ed-42f4-a607-49306de661bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.503283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9efec0-25ed-42f4-a607-49306de661bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:23:19.503606 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9efec0-25ed-42f4-a607-49306de661bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:19.504657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: WARNING py.warnings [None req-5a9efec0-25ed-42f4-a607-49306de661bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:23:19.504657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: warnings.warn(msg) Sep 19 11:23:19.504657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: Sep 19 11:23:19.505537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9efec0-25ed-42f4-a607-49306de661bc admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:19.512789 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 761/1547] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:19.513661 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d197e0-32d2-4eae-aaf3-d86a63c384ba None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:19.517878 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 787/1548] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:23:19.519138 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feed9a7-951a-47fc-8679-f1e89d94489a None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.519422 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feed9a7-951a-47fc-8679-f1e89d94489a None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.519691 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feed9a7-951a-47fc-8679-f1e89d94489a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.523703 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d197e0-32d2-4eae-aaf3-d86a63c384ba None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:19.528911 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80970e-18e5-4c7a-a52f-092f77cba114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:19.529510 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80970e-18e5-4c7a-a52f-092f77cba114 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.529794 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80970e-18e5-4c7a-a52f-092f77cba114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.530064 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80970e-18e5-4c7a-a52f-092f77cba114 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.530694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c80970e-18e5-4c7a-a52f-092f77cba114 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:19.532683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c80970e-18e5-4c7a-a52f-092f77cba114 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:19.539014 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-4feed9a7-951a-47fc-8679-f1e89d94489a None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:19.540238 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-4feed9a7-951a-47fc-8679-f1e89d94489a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:19.549264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 788/1549] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:19.559943 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 762/1550] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/users => generated 302 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:19.559943 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d780a9f-87a4-49ce-bb26-9425b09fd8cb None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.560264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cd3b6-f080-4eda-bbf6-23a962edbb6c None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.560264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cd3b6-f080-4eda-bbf6-23a962edbb6c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.560429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d780a9f-87a4-49ce-bb26-9425b09fd8cb None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.560523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cd3b6-f080-4eda-bbf6-23a962edbb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.560614 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d780a9f-87a4-49ce-bb26-9425b09fd8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.578421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6d780a9f-87a4-49ce-bb26-9425b09fd8cb None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:19.578611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-f19cd3b6-f080-4eda-bbf6-23a962edbb6c None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:19.579616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6d780a9f-87a4-49ce-bb26-9425b09fd8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:19.579783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f19cd3b6-f080-4eda-bbf6-23a962edbb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:19.587986 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 763/1552] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:19.588117 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 789/1552] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:19.589500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad8ad6-8e46-4308-90a8-e052557b4b4e None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.589782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad8ad6-8e46-4308-90a8-e052557b4b4e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.590053 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad8ad6-8e46-4308-90a8-e052557b4b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.593256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4f526-9aa6-46a8-a29c-c9e4a60a16ca None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.593597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4f526-9aa6-46a8-a29c-c9e4a60a16ca None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.593924 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4f526-9aa6-46a8-a29c-c9e4a60a16ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.612054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-6bad8ad6-8e46-4308-90a8-e052557b4b4e None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:19.613254 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-6bad8ad6-8e46-4308-90a8-e052557b4b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:19.614094 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-29c4f526-9aa6-46a8-a29c-c9e4a60a16ca None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:19.615250 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-29c4f526-9aa6-46a8-a29c-c9e4a60a16ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:19.621346 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 790/1553] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:19.622049 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad8ad6-8e46-4308-90a8-e052557b4b4e None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:19.623813 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 764/1554] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:19.628581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4f526-9aa6-46a8-a29c-c9e4a60a16ca None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:19.633945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad8ad6-8e46-4308-90a8-e052557b4b4e None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:19.638638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f196ed-53c1-4fa7-9fef-d1cc36a0bf15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:19.639085 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4f526-9aa6-46a8-a29c-c9e4a60a16ca None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:19.639227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f196ed-53c1-4fa7-9fef-d1cc36a0bf15 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.639507 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f196ed-53c1-4fa7-9fef-d1cc36a0bf15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.639797 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f196ed-53c1-4fa7-9fef-d1cc36a0bf15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.640442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f196ed-53c1-4fa7-9fef-d1cc36a0bf15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:23:19.640858 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f196ed-53c1-4fa7-9fef-d1cc36a0bf15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:19.642449 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f196ed-53c1-4fa7-9fef-d1cc36a0bf15 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:19.643810 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8faee-0cb5-4e8b-b80e-d5ea396c04a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:19.644407 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8faee-0cb5-4e8b-b80e-d5ea396c04a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.644689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8faee-0cb5-4e8b-b80e-d5ea396c04a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.645318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8faee-0cb5-4e8b-b80e-d5ea396c04a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.645585 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a8faee-0cb5-4e8b-b80e-d5ea396c04a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:19.646599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: WARNING py.warnings [None req-c5a8faee-0cb5-4e8b-b80e-d5ea396c04a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:23:19.646599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: warnings.warn(msg) Sep 19 11:23:19.646599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: Sep 19 11:23:19.647799 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a8faee-0cb5-4e8b-b80e-d5ea396c04a1 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:19.655145 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 791/1555] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:23:19.656403 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279e5e1-ae83-497c-abad-0c2378c77cab None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.656760 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279e5e1-ae83-497c-abad-0c2378c77cab None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.657030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279e5e1-ae83-497c-abad-0c2378c77cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.677370 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-f279e5e1-ae83-497c-abad-0c2378c77cab None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:19.678904 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f279e5e1-ae83-497c-abad-0c2378c77cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:19.687584 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 792/1556] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:19.688399 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279e5e1-ae83-497c-abad-0c2378c77cab None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:19.692242 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 765/1557] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:19.692242 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9576674-6e95-452c-b082-efcdf7d7a75d None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.692242 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9576674-6e95-452c-b082-efcdf7d7a75d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.692242 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9576674-6e95-452c-b082-efcdf7d7a75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.697887 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279e5e1-ae83-497c-abad-0c2378c77cab None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:19.711484 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-b9576674-6e95-452c-b082-efcdf7d7a75d None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:19.712733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b9576674-6e95-452c-b082-efcdf7d7a75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:19.756504 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e361e7-918d-44cd-b77c-aa4459ccacb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:19.757116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e361e7-918d-44cd-b77c-aa4459ccacb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.757403 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e361e7-918d-44cd-b77c-aa4459ccacb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.757672 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e361e7-918d-44cd-b77c-aa4459ccacb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.758317 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e361e7-918d-44cd-b77c-aa4459ccacb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:19.760012 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e361e7-918d-44cd-b77c-aa4459ccacb3 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:19.770163 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 793/1558] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:19.771719 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1257995-f70b-4e54-97d0-2aecf4f3d7f7 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.772216 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1257995-f70b-4e54-97d0-2aecf4f3d7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.772682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1257995-f70b-4e54-97d0-2aecf4f3d7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.779193 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 766/1559] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:19.781980 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239d83d-f286-463d-b602-671021250a07 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.782278 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239d83d-f286-463d-b602-671021250a07 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.782558 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239d83d-f286-463d-b602-671021250a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.792950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:19.799655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-b1257995-f70b-4e54-97d0-2aecf4f3d7f7 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:19.801450 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b1257995-f70b-4e54-97d0-2aecf4f3d7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:19.804460 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6239d83d-f286-463d-b602-671021250a07 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:19.805661 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6239d83d-f286-463d-b602-671021250a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:19.810040 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 794/1560] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:19.811228 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c1cc6-e500-4177-a5f3-8493e9b89477 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.811514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c1cc6-e500-4177-a5f3-8493e9b89477 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.811816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c1cc6-e500-4177-a5f3-8493e9b89477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.817840 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 767/1561] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:19.818600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239d83d-f286-463d-b602-671021250a07 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:19.827978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239d83d-f286-463d-b602-671021250a07 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:19.832226 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-d86c1cc6-e500-4177-a5f3-8493e9b89477 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:19.832474 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ba2bd-ad79-48a0-9824-b361409768ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:19.833055 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ba2bd-ad79-48a0-9824-b361409768ac admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.833334 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ba2bd-ad79-48a0-9824-b361409768ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.833466 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-d86c1cc6-e500-4177-a5f3-8493e9b89477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:19.833639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ba2bd-ad79-48a0-9824-b361409768ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.834518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ba2bd-ad79-48a0-9824-b361409768ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:19.836429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ba2bd-ad79-48a0-9824-b361409768ac admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:19.842185 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:19.842659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 795/1562] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:19.843521 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c1cc6-e500-4177-a5f3-8493e9b89477 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:19.853846 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c1cc6-e500-4177-a5f3-8493e9b89477 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:19.859471 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b812d8-d0fd-4cc4-9b95-f1b871a64565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:19.860309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b812d8-d0fd-4cc4-9b95-f1b871a64565 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.860596 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b812d8-d0fd-4cc4-9b95-f1b871a64565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.860869 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b812d8-d0fd-4cc4-9b95-f1b871a64565 admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.861512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b812d8-d0fd-4cc4-9b95-f1b871a64565 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:23:19.861816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b812d8-d0fd-4cc4-9b95-f1b871a64565 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:19.863442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b812d8-d0fd-4cc4-9b95-f1b871a64565 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:19.866244 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 768/1563] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:19.867064 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ba2bd-ad79-48a0-9824-b361409768ac admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:19.878064 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 796/1564] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:23:19.878823 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b812d8-d0fd-4cc4-9b95-f1b871a64565 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:19.889288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b812d8-d0fd-4cc4-9b95-f1b871a64565 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:19.929087 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ba2bd-ad79-48a0-9824-b361409768ac admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:19.934187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a34d55-b204-4c38-8b7b-53af36a0ffac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:19.934787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a34d55-b204-4c38-8b7b-53af36a0ffac admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.935063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a34d55-b204-4c38-8b7b-53af36a0ffac admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.935330 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a34d55-b204-4c38-8b7b-53af36a0ffac admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.936095 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a34d55-b204-4c38-8b7b-53af36a0ffac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:19.938065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a34d55-b204-4c38-8b7b-53af36a0ffac admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:19.945292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ca819-d7af-4a54-aea3-bf2aca2d5507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:19.945892 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ca819-d7af-4a54-aea3-bf2aca2d5507 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.946177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ca819-d7af-4a54-aea3-bf2aca2d5507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.946470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ca819-d7af-4a54-aea3-bf2aca2d5507 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.947105 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ca819-d7af-4a54-aea3-bf2aca2d5507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:19.949564 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ca819-d7af-4a54-aea3-bf2aca2d5507 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:19.964131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 769/1565] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:19 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:19.964333 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 797/1566] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:19.964444 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a34d55-b204-4c38-8b7b-53af36a0ffac admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:19.964560 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ca819-d7af-4a54-aea3-bf2aca2d5507 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:19.972087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ca819-d7af-4a54-aea3-bf2aca2d5507 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:19.974542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a34d55-b204-4c38-8b7b-53af36a0ffac admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:19.977044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7a899-40da-4ff7-a26d-dba05c029cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:19.977799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7a899-40da-4ff7-a26d-dba05c029cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.978205 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7a899-40da-4ff7-a26d-dba05c029cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.978483 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7a899-40da-4ff7-a26d-dba05c029cf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.979641 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c7a899-40da-4ff7-a26d-dba05c029cf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:23:19.979816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32904b-3270-4749-926f-37a1096950a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:19.979967 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c7a899-40da-4ff7-a26d-dba05c029cf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:19.980405 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32904b-3270-4749-926f-37a1096950a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:19.980686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32904b-3270-4749-926f-37a1096950a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:19.980959 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32904b-3270-4749-926f-37a1096950a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:19.981559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c7a899-40da-4ff7-a26d-dba05c029cf4 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:19.981704 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c32904b-3270-4749-926f-37a1096950a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:23:19.981880 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c32904b-3270-4749-926f-37a1096950a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:19.983473 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c32904b-3270-4749-926f-37a1096950a6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:19.993518 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 798/1567] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:23:19.994299 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7a899-40da-4ff7-a26d-dba05c029cf4 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.003183 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 770/1568] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:23:20.003183 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32904b-3270-4749-926f-37a1096950a6 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.005017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7a899-40da-4ff7-a26d-dba05c029cf4 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.010314 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9525dc-734f-4210-97c2-ee0e9be0061d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.010918 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9525dc-734f-4210-97c2-ee0e9be0061d admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.011194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9525dc-734f-4210-97c2-ee0e9be0061d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.011462 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9525dc-734f-4210-97c2-ee0e9be0061d admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.012122 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9525dc-734f-4210-97c2-ee0e9be0061d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:23:20.012416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9525dc-734f-4210-97c2-ee0e9be0061d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.012526 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32904b-3270-4749-926f-37a1096950a6 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.013975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9525dc-734f-4210-97c2-ee0e9be0061d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.017104 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3fde8-d395-41ed-a8be-9d8290282580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.018170 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3fde8-d395-41ed-a8be-9d8290282580 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.018452 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3fde8-d395-41ed-a8be-9d8290282580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.018858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3fde8-d395-41ed-a8be-9d8290282580 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.019513 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3fde8-d395-41ed-a8be-9d8290282580 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:23:20.019860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3fde8-d395-41ed-a8be-9d8290282580 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.021454 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3fde8-d395-41ed-a8be-9d8290282580 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.025255 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 799/1569] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:23:20.026095 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9525dc-734f-4210-97c2-ee0e9be0061d admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.033640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 771/1570] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:23:20.034444 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3fde8-d395-41ed-a8be-9d8290282580 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.036275 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9525dc-734f-4210-97c2-ee0e9be0061d admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.041200 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2eee2c-363d-4a03-a0ec-caa37260d025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.042289 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2eee2c-363d-4a03-a0ec-caa37260d025 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.042565 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2eee2c-363d-4a03-a0ec-caa37260d025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.042836 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2eee2c-363d-4a03-a0ec-caa37260d025 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.043453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2eee2c-363d-4a03-a0ec-caa37260d025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.044520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: WARNING py.warnings [None req-7f2eee2c-363d-4a03-a0ec-caa37260d025 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:23:20.044520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: warnings.warn(msg) Sep 19 11:23:20.044520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: Sep 19 11:23:20.045712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2eee2c-363d-4a03-a0ec-caa37260d025 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.045820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3fde8-d395-41ed-a8be-9d8290282580 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.050648 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a57d88-6153-408d-8e55-0cf54d9433f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.051236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a57d88-6153-408d-8e55-0cf54d9433f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.051518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a57d88-6153-408d-8e55-0cf54d9433f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.051824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a57d88-6153-408d-8e55-0cf54d9433f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.052452 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a57d88-6153-408d-8e55-0cf54d9433f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.054364 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a57d88-6153-408d-8e55-0cf54d9433f8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.067797 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 800/1571] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:20 2022] POST /identity/v3/users => generated 288 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:20.068417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2eee2c-363d-4a03-a0ec-caa37260d025 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.078840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2eee2c-363d-4a03-a0ec-caa37260d025 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.083518 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87235c97-bc4e-40a2-9101-7432987803fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.084441 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87235c97-bc4e-40a2-9101-7432987803fa admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.084712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87235c97-bc4e-40a2-9101-7432987803fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.084988 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87235c97-bc4e-40a2-9101-7432987803fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.085623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87235c97-bc4e-40a2-9101-7432987803fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.089581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 772/1572] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:20 2022] POST /identity/v3/projects => generated 393 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:20.089581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a57d88-6153-408d-8e55-0cf54d9433f8 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.089807 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87235c97-bc4e-40a2-9101-7432987803fa admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.100815 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 801/1573] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:20.100815 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87235c97-bc4e-40a2-9101-7432987803fa admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.108686 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87235c97-bc4e-40a2-9101-7432987803fa admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.154980 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a57d88-6153-408d-8e55-0cf54d9433f8 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.159846 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac227b-f3d9-458a-b643-025ce12f82b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.160499 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac227b-f3d9-458a-b643-025ce12f82b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.160775 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac227b-f3d9-458a-b643-025ce12f82b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.161043 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac227b-f3d9-458a-b643-025ce12f82b5 admin admin] PATH_INFO: `/v3/projects/cfccfb243e074da4958a5c4855232cf1/users/53d79984cadc4fb2a477e05c5c7a7f11/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.174437 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f043bbd-83de-44eb-9bd7-422601ca95db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.175040 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f043bbd-83de-44eb-9bd7-422601ca95db admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.175322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f043bbd-83de-44eb-9bd7-422601ca95db admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.175602 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f043bbd-83de-44eb-9bd7-422601ca95db admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.176255 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f043bbd-83de-44eb-9bd7-422601ca95db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.176847 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac227b-f3d9-458a-b643-025ce12f82b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfccfb243e074da4958a5c4855232cf1, user_id=53d79984cadc4fb2a477e05c5c7a7f11, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.178018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: WARNING py.warnings [None req-5cac227b-f3d9-458a-b643-025ce12f82b5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:23:20.178018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: warnings.warn(msg) Sep 19 11:23:20.178018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: Sep 19 11:23:20.178279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f043bbd-83de-44eb-9bd7-422601ca95db admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.180023 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac227b-f3d9-458a-b643-025ce12f82b5 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.193684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 773/1574] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:20 2022] PUT /identity/v3/projects/cfccfb243e074da4958a5c4855232cf1/users/53d79984cadc4fb2a477e05c5c7a7f11/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:20.196709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac227b-f3d9-458a-b643-025ce12f82b5 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.210750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 802/1575] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:20 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:20.210750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f043bbd-83de-44eb-9bd7-422601ca95db admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.215394 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:20.218336 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:20.266194 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac227b-f3d9-458a-b643-025ce12f82b5 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.270965 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd5a0e-5d25-48d2-a1e2-98a6f5729484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.271571 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd5a0e-5d25-48d2-a1e2-98a6f5729484 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.271878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd5a0e-5d25-48d2-a1e2-98a6f5729484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.272152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd5a0e-5d25-48d2-a1e2-98a6f5729484 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.272782 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd5a0e-5d25-48d2-a1e2-98a6f5729484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.273833 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f043bbd-83de-44eb-9bd7-422601ca95db admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.274697 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd5a0e-5d25-48d2-a1e2-98a6f5729484 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.278779 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fa7dd-9103-4a47-986d-93ccfa3d6b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.279648 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fa7dd-9103-4a47-986d-93ccfa3d6b78 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.280130 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fa7dd-9103-4a47-986d-93ccfa3d6b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.280398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fa7dd-9103-4a47-986d-93ccfa3d6b78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.281022 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fa7dd-9103-4a47-986d-93ccfa3d6b78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.282915 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fa7dd-9103-4a47-986d-93ccfa3d6b78 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.302820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 774/1576] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:20 2022] POST /identity/v3/projects => generated 355 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:20.303725 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd5a0e-5d25-48d2-a1e2-98a6f5729484 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.313932 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 803/1577] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:20 2022] POST /identity/v3/projects => generated 367 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:20.314793 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fa7dd-9103-4a47-986d-93ccfa3d6b78 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.371828 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd5a0e-5d25-48d2-a1e2-98a6f5729484 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.376315 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fa7dd-9103-4a47-986d-93ccfa3d6b78 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.376481 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebfd9f-d510-40a3-bad1-9230058c89c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.377136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebfd9f-d510-40a3-bad1-9230058c89c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.377416 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebfd9f-d510-40a3-bad1-9230058c89c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.377690 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebfd9f-d510-40a3-bad1-9230058c89c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.378319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeebfd9f-d510-40a3-bad1-9230058c89c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.380269 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeebfd9f-d510-40a3-bad1-9230058c89c0 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.381247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558495e-85f2-40f6-8461-b34fab7324b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.382034 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558495e-85f2-40f6-8461-b34fab7324b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.382392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558495e-85f2-40f6-8461-b34fab7324b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.382736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558495e-85f2-40f6-8461-b34fab7324b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.383441 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558495e-85f2-40f6-8461-b34fab7324b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.385178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558495e-85f2-40f6-8461-b34fab7324b2 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.395208 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 804/1578] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:20.396218 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558495e-85f2-40f6-8461-b34fab7324b2 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.407039 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558495e-85f2-40f6-8461-b34fab7324b2 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.412698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 775/1579] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:20 2022] POST /identity/v3/projects => generated 389 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:20.412698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebfd9f-d510-40a3-bad1-9230058c89c0 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.412943 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75ff6c-c151-4172-a9bf-8c87bb46697e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.412943 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75ff6c-c151-4172-a9bf-8c87bb46697e admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.412943 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75ff6c-c151-4172-a9bf-8c87bb46697e admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.412943 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75ff6c-c151-4172-a9bf-8c87bb46697e admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.413476 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad75ff6c-c151-4172-a9bf-8c87bb46697e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.415363 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad75ff6c-c151-4172-a9bf-8c87bb46697e admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.441309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 805/1580] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:20 2022] POST /identity/v3/users => generated 297 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:20.441309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94133770-4474-467f-b1a1-d8c3eaba5529 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.441309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94133770-4474-467f-b1a1-d8c3eaba5529 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.441309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94133770-4474-467f-b1a1-d8c3eaba5529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.468577 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-94133770-4474-467f-b1a1-d8c3eaba5529 None None] MFA Rules not processed for user `53d79984cadc4fb2a477e05c5c7a7f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:20.469802 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-94133770-4474-467f-b1a1-d8c3eaba5529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:20.478446 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebfd9f-d510-40a3-bad1-9230058c89c0 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.483615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2085ae-69f6-4eb8-9f7a-d3b088a7d2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.484255 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2085ae-69f6-4eb8-9f7a-d3b088a7d2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.484519 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2085ae-69f6-4eb8-9f7a-d3b088a7d2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.484790 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2085ae-69f6-4eb8-9f7a-d3b088a7d2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.485422 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2085ae-69f6-4eb8-9f7a-d3b088a7d2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.487025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2085ae-69f6-4eb8-9f7a-d3b088a7d2e4 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.496913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 776/1581] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:20.497652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2085ae-69f6-4eb8-9f7a-d3b088a7d2e4 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.507954 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2085ae-69f6-4eb8-9f7a-d3b088a7d2e4 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.513359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1afbf-0e63-403e-b7f1-f6e5213f114f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.514240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1afbf-0e63-403e-b7f1-f6e5213f114f admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.514532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1afbf-0e63-403e-b7f1-f6e5213f114f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.514826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1afbf-0e63-403e-b7f1-f6e5213f114f admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.515434 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1afbf-0e63-403e-b7f1-f6e5213f114f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.517889 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1afbf-0e63-403e-b7f1-f6e5213f114f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.533892 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:20.535807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:20.540091 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 806/1582] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:20 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:20.541152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94133770-4474-467f-b1a1-d8c3eaba5529 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.541564 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 777/1583] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:20 2022] POST /identity/v3/users => generated 287 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:20.542933 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1afbf-0e63-403e-b7f1-f6e5213f114f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.553948 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1afbf-0e63-403e-b7f1-f6e5213f114f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.554488 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94133770-4474-467f-b1a1-d8c3eaba5529 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.558625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec61bdf-2367-4444-8a19-c327cf5bbcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.559210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec61bdf-2367-4444-8a19-c327cf5bbcc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.559351 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a484869-7b3a-42c0-be6c-74db422caebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.559490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec61bdf-2367-4444-8a19-c327cf5bbcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.559779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec61bdf-2367-4444-8a19-c327cf5bbcc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.559967 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a484869-7b3a-42c0-be6c-74db422caebf admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.560247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a484869-7b3a-42c0-be6c-74db422caebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.560414 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec61bdf-2367-4444-8a19-c327cf5bbcc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.560524 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a484869-7b3a-42c0-be6c-74db422caebf admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.561126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a484869-7b3a-42c0-be6c-74db422caebf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.562329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec61bdf-2367-4444-8a19-c327cf5bbcc8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.563015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a484869-7b3a-42c0-be6c-74db422caebf admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.589239 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 778/1584] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:20 2022] POST /identity/v3/users => generated 284 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:20.589441 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 807/1585] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:20 2022] POST /identity/v3/users => generated 278 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:20.589441 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a484869-7b3a-42c0-be6c-74db422caebf admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.590250 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec61bdf-2367-4444-8a19-c327cf5bbcc8 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.598577 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a484869-7b3a-42c0-be6c-74db422caebf admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.599677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec61bdf-2367-4444-8a19-c327cf5bbcc8 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.603175 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3149740-2050-4fe0-9cbf-29f7167d3410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.603807 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3149740-2050-4fe0-9cbf-29f7167d3410 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.604087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3149740-2050-4fe0-9cbf-29f7167d3410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.604792 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74114a6-1ed6-4939-8144-32f7e237114f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.605058 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3149740-2050-4fe0-9cbf-29f7167d3410 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.605375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74114a6-1ed6-4939-8144-32f7e237114f admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.605652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74114a6-1ed6-4939-8144-32f7e237114f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.605761 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3149740-2050-4fe0-9cbf-29f7167d3410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.605920 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74114a6-1ed6-4939-8144-32f7e237114f admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.606553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74114a6-1ed6-4939-8144-32f7e237114f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.607560 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3149740-2050-4fe0-9cbf-29f7167d3410 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.608175 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74114a6-1ed6-4939-8144-32f7e237114f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.618390 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 779/1586] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:20.619330 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74114a6-1ed6-4939-8144-32f7e237114f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.629366 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74114a6-1ed6-4939-8144-32f7e237114f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.631151 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 808/1587] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:20 2022] POST /identity/v3/users => generated 295 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:20.632803 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3149740-2050-4fe0-9cbf-29f7167d3410 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.634405 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fd8c2-ed17-4f76-b64b-2120ee6ffe1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.634988 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fd8c2-ed17-4f76-b64b-2120ee6ffe1d admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.635269 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fd8c2-ed17-4f76-b64b-2120ee6ffe1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.635598 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fd8c2-ed17-4f76-b64b-2120ee6ffe1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.636259 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fd8c2-ed17-4f76-b64b-2120ee6ffe1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.637841 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fd8c2-ed17-4f76-b64b-2120ee6ffe1d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.643011 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3149740-2050-4fe0-9cbf-29f7167d3410 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.650718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 780/1588] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:20.650867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e41315-d6c7-4f62-a293-179dcb97bc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.651523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fd8c2-ed17-4f76-b64b-2120ee6ffe1d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.651618 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e41315-d6c7-4f62-a293-179dcb97bc38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.651618 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e41315-d6c7-4f62-a293-179dcb97bc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.651618 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e41315-d6c7-4f62-a293-179dcb97bc38 admin admin] PATH_INFO: `/v3/projects/8fd848b22f854563a0ab63ec0971b595/users/aada82b4250745ab956466f29d1ddfb1/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.659684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fd8c2-ed17-4f76-b64b-2120ee6ffe1d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.664263 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9b38a-16a7-407b-a853-d1bd1987c14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.664849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9b38a-16a7-407b-a853-d1bd1987c14a admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.665127 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9b38a-16a7-407b-a853-d1bd1987c14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.665396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9b38a-16a7-407b-a853-d1bd1987c14a admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.666019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9b38a-16a7-407b-a853-d1bd1987c14a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.666231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e41315-d6c7-4f62-a293-179dcb97bc38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd848b22f854563a0ab63ec0971b595, user_id=aada82b4250745ab956466f29d1ddfb1, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.668934 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9b38a-16a7-407b-a853-d1bd1987c14a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.669574 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e41315-d6c7-4f62-a293-179dcb97bc38 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.682993 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 809/1589] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:20 2022] PUT /identity/v3/projects/8fd848b22f854563a0ab63ec0971b595/users/aada82b4250745ab956466f29d1ddfb1/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:20.683991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e41315-d6c7-4f62-a293-179dcb97bc38 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.699973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 781/1590] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:20 2022] POST /identity/v3/projects => generated 405 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:20.701044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9b38a-16a7-407b-a853-d1bd1987c14a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.749811 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e41315-d6c7-4f62-a293-179dcb97bc38 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.762984 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9b38a-16a7-407b-a853-d1bd1987c14a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.767641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34a0cd-b18f-49ad-a81c-72b01339a1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.768275 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34a0cd-b18f-49ad-a81c-72b01339a1e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.768683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34a0cd-b18f-49ad-a81c-72b01339a1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.769000 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34a0cd-b18f-49ad-a81c-72b01339a1e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.769697 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34a0cd-b18f-49ad-a81c-72b01339a1e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.771330 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34a0cd-b18f-49ad-a81c-72b01339a1e2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.782048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 782/1591] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:20.782934 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34a0cd-b18f-49ad-a81c-72b01339a1e2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.792413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34a0cd-b18f-49ad-a81c-72b01339a1e2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.796922 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45155a26-5b6a-4650-bd87-ae09126a70b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.797489 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45155a26-5b6a-4650-bd87-ae09126a70b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.797770 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45155a26-5b6a-4650-bd87-ae09126a70b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.798043 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45155a26-5b6a-4650-bd87-ae09126a70b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.798664 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45155a26-5b6a-4650-bd87-ae09126a70b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.800342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45155a26-5b6a-4650-bd87-ae09126a70b6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.809935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 783/1592] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:20.810757 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45155a26-5b6a-4650-bd87-ae09126a70b6 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.811747 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88754a13-858f-43d3-8f6f-3421ce808aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.812409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88754a13-858f-43d3-8f6f-3421ce808aab admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.812703 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88754a13-858f-43d3-8f6f-3421ce808aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.813174 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88754a13-858f-43d3-8f6f-3421ce808aab admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.813593 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88754a13-858f-43d3-8f6f-3421ce808aab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.815194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88754a13-858f-43d3-8f6f-3421ce808aab admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.820343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45155a26-5b6a-4650-bd87-ae09126a70b6 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.825820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fe30d-2bac-4f3a-acce-af0e73704f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.829178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 810/1593] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:20.829308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fe30d-2bac-4f3a-acce-af0e73704f8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.829308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fe30d-2bac-4f3a-acce-af0e73704f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.829308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fe30d-2bac-4f3a-acce-af0e73704f8a admin admin] PATH_INFO: `/v3/projects/e8a952d8324548d7a8e80f72449ea5e6/users/9a4a7aca62974db38138a38de399139f/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.829601 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88754a13-858f-43d3-8f6f-3421ce808aab admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.836706 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88754a13-858f-43d3-8f6f-3421ce808aab admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.842014 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d4b5f-419d-4f19-9e00-8143cb61b27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.842586 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d4b5f-419d-4f19-9e00-8143cb61b27e admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.842860 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d4b5f-419d-4f19-9e00-8143cb61b27e admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.843110 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fe30d-2bac-4f3a-acce-af0e73704f8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8a952d8324548d7a8e80f72449ea5e6, user_id=9a4a7aca62974db38138a38de399139f, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.843212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d4b5f-419d-4f19-9e00-8143cb61b27e admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.843744 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d4b5f-419d-4f19-9e00-8143cb61b27e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.845309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d4b5f-419d-4f19-9e00-8143cb61b27e admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.845822 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fe30d-2bac-4f3a-acce-af0e73704f8a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.855847 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 811/1594] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:20.855847 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d4b5f-419d-4f19-9e00-8143cb61b27e admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.860189 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 784/1595] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:20 2022] PUT /identity/v3/projects/e8a952d8324548d7a8e80f72449ea5e6/users/9a4a7aca62974db38138a38de399139f/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:20.861548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e0ab6-f953-418b-b806-1ad5d9909f88 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.861838 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e0ab6-f953-418b-b806-1ad5d9909f88 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.862263 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e0ab6-f953-418b-b806-1ad5d9909f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.870678 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d4b5f-419d-4f19-9e00-8143cb61b27e admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.882219 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-590e0ab6-f953-418b-b806-1ad5d9909f88 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:20.883438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-590e0ab6-f953-418b-b806-1ad5d9909f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:20.900408 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:20.930270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25157bc4-1f03-44c2-ba8b-7d16b37a1f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.930792 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25157bc4-1f03-44c2-ba8b-7d16b37a1f40 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.931068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25157bc4-1f03-44c2-ba8b-7d16b37a1f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.931337 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25157bc4-1f03-44c2-ba8b-7d16b37a1f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.931999 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25157bc4-1f03-44c2-ba8b-7d16b37a1f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.934213 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25157bc4-1f03-44c2-ba8b-7d16b37a1f40 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.937305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:20.944605 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 812/1596] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:20.945375 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25157bc4-1f03-44c2-ba8b-7d16b37a1f40 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.947064 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 785/1597] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:20.947830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e0ab6-f953-418b-b806-1ad5d9909f88 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.958154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e0ab6-f953-418b-b806-1ad5d9909f88 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.958694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25157bc4-1f03-44c2-ba8b-7d16b37a1f40 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.964581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e1c66-0202-41b8-b741-e63acbf57069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.964975 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60fc52-6208-41a6-9d9a-317e790313fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.965179 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e1c66-0202-41b8-b741-e63acbf57069 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.965461 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e1c66-0202-41b8-b741-e63acbf57069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.965572 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60fc52-6208-41a6-9d9a-317e790313fa admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.965736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e1c66-0202-41b8-b741-e63acbf57069 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.965837 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60fc52-6208-41a6-9d9a-317e790313fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.966093 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60fc52-6208-41a6-9d9a-317e790313fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.966397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e1c66-0202-41b8-b741-e63acbf57069 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.966740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60fc52-6208-41a6-9d9a-317e790313fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:20.968358 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60fc52-6208-41a6-9d9a-317e790313fa admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.968497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e1c66-0202-41b8-b741-e63acbf57069 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:20.978348 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 786/1598] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:20.979675 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60fc52-6208-41a6-9d9a-317e790313fa admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.989171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60fc52-6208-41a6-9d9a-317e790313fa admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:20.993395 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 813/1599] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:20 2022] POST /identity/v3/users => generated 309 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:20.994178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e1c66-0202-41b8-b741-e63acbf57069 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:20.996413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac1d24-c8ea-49ab-9b8c-0e3ef5546351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:20.997139 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac1d24-c8ea-49ab-9b8c-0e3ef5546351 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:20.997424 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac1d24-c8ea-49ab-9b8c-0e3ef5546351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:20.997697 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac1d24-c8ea-49ab-9b8c-0e3ef5546351 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:20.998372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac1d24-c8ea-49ab-9b8c-0e3ef5546351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:21.000059 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac1d24-c8ea-49ab-9b8c-0e3ef5546351 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:21.004043 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e1c66-0202-41b8-b741-e63acbf57069 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:21.008590 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33774bfc-8ba2-4398-9d43-b0e7895088ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:21.009164 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33774bfc-8ba2-4398-9d43-b0e7895088ed admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:21.009439 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33774bfc-8ba2-4398-9d43-b0e7895088ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:21.009709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33774bfc-8ba2-4398-9d43-b0e7895088ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:21.010371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33774bfc-8ba2-4398-9d43-b0e7895088ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:21.010506 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 787/1600] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:21.011454 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac1d24-c8ea-49ab-9b8c-0e3ef5546351 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:21.012007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33774bfc-8ba2-4398-9d43-b0e7895088ed admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:21.021153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac1d24-c8ea-49ab-9b8c-0e3ef5546351 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:21.022903 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 814/1601] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:21.024108 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d96b4-fa66-4e25-829d-0e2a5bcc6e49 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:21.024393 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d96b4-fa66-4e25-829d-0e2a5bcc6e49 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:21.024681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d96b4-fa66-4e25-829d-0e2a5bcc6e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:21.029044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771a7de-f787-4af7-88b2-c70ed73b761d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:21.029630 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771a7de-f787-4af7-88b2-c70ed73b761d admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:21.029915 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771a7de-f787-4af7-88b2-c70ed73b761d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:21.030188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771a7de-f787-4af7-88b2-c70ed73b761d admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:21.030821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771a7de-f787-4af7-88b2-c70ed73b761d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:21.032447 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771a7de-f787-4af7-88b2-c70ed73b761d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:21.042947 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-5d8d96b4-fa66-4e25-829d-0e2a5bcc6e49 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:21.043301 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 788/1602] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:21.044157 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771a7de-f787-4af7-88b2-c70ed73b761d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:21.044257 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5d8d96b4-fa66-4e25-829d-0e2a5bcc6e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:21.054796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 815/1603] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:21.054796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d96b4-fa66-4e25-829d-0e2a5bcc6e49 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:21.055044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771a7de-f787-4af7-88b2-c70ed73b761d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:21.059631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776266e-9240-4a72-96be-cd73852a8ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:21.060446 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776266e-9240-4a72-96be-cd73852a8ab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:21.060845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776266e-9240-4a72-96be-cd73852a8ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:21.061196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776266e-9240-4a72-96be-cd73852a8ab0 admin admin] PATH_INFO: `/v3/projects/c7ccdce201f547f5a50c93e27c5e8cb1/users/9f160368e1984319ad4fa37c2794a956/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:21.076473 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c776266e-9240-4a72-96be-cd73852a8ab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7ccdce201f547f5a50c93e27c5e8cb1, user_id=9f160368e1984319ad4fa37c2794a956, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:21.079164 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c776266e-9240-4a72-96be-cd73852a8ab0 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:21.093676 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 789/1604] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:21 2022] PUT /identity/v3/projects/c7ccdce201f547f5a50c93e27c5e8cb1/users/9f160368e1984319ad4fa37c2794a956/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:21.094447 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776266e-9240-4a72-96be-cd73852a8ab0 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:21.114433 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d96b4-fa66-4e25-829d-0e2a5bcc6e49 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:21.158920 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776266e-9240-4a72-96be-cd73852a8ab0 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:21.163882 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbded347-34d7-43b8-9fe1-2edbb7bfbc61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:21.164500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbded347-34d7-43b8-9fe1-2edbb7bfbc61 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:21.164664 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520e634-86f0-42cb-8c21-459b6558fd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:21.164783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbded347-34d7-43b8-9fe1-2edbb7bfbc61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:21.165043 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbded347-34d7-43b8-9fe1-2edbb7bfbc61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:21.165224 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520e634-86f0-42cb-8c21-459b6558fd10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:21.165501 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520e634-86f0-42cb-8c21-459b6558fd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:21.165773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520e634-86f0-42cb-8c21-459b6558fd10 admin admin] PATH_INFO: `/v3/projects/62cdba2a8ecf40c7b320b495b3bf6c7d/users/e1546ba5c727467c845925be6dbc4ec0/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:21.174959 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbded347-34d7-43b8-9fe1-2edbb7bfbc61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:21.176453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbded347-34d7-43b8-9fe1-2edbb7bfbc61 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:21.180681 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520e634-86f0-42cb-8c21-459b6558fd10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62cdba2a8ecf40c7b320b495b3bf6c7d, user_id=e1546ba5c727467c845925be6dbc4ec0, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:21.183823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520e634-86f0-42cb-8c21-459b6558fd10 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:21.186804 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 816/1605] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:21.188625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbded347-34d7-43b8-9fe1-2edbb7bfbc61 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:21.197843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 790/1606] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:21 2022] PUT /identity/v3/projects/62cdba2a8ecf40c7b320b495b3bf6c7d/users/e1546ba5c727467c845925be6dbc4ec0/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:21.209321 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbded347-34d7-43b8-9fe1-2edbb7bfbc61 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:21.209457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520e634-86f0-42cb-8c21-459b6558fd10 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:21.252909 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:21.266459 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4979c-27c8-4e3b-9de4-bfa00fa2edbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:21.267538 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4979c-27c8-4e3b-9de4-bfa00fa2edbf admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:21.268233 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4979c-27c8-4e3b-9de4-bfa00fa2edbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:21.268814 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4979c-27c8-4e3b-9de4-bfa00fa2edbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:21.269562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4979c-27c8-4e3b-9de4-bfa00fa2edbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:21.271340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b4979c-27c8-4e3b-9de4-bfa00fa2edbf admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:21.283309 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520e634-86f0-42cb-8c21-459b6558fd10 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:21.288649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 817/1607] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:21.289786 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4979c-27c8-4e3b-9de4-bfa00fa2edbf admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:21.293501 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4c620-b584-47b3-afa0-a65ab4316315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:21.294373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4c620-b584-47b3-afa0-a65ab4316315 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:21.294662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4c620-b584-47b3-afa0-a65ab4316315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:21.295238 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4c620-b584-47b3-afa0-a65ab4316315 admin admin] PATH_INFO: `/v3/projects/dcb15b699d5f4db990b0f9fd451eb470/users/33b8d13b149c4af48ead3d6650ee7ec5/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:21.301639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b4979c-27c8-4e3b-9de4-bfa00fa2edbf admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:21.306696 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa029f3-d44e-4af8-93ae-bd8aa0e205d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:21.307504 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa029f3-d44e-4af8-93ae-bd8aa0e205d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:21.307931 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa029f3-d44e-4af8-93ae-bd8aa0e205d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:21.308307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa029f3-d44e-4af8-93ae-bd8aa0e205d2 admin admin] PATH_INFO: `/v3/projects/2894018d1d984b839e5b900a1be2e896/users/bca0348930b94e289519b645b8ec8458/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:21.311695 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4c620-b584-47b3-afa0-a65ab4316315 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcb15b699d5f4db990b0f9fd451eb470, user_id=33b8d13b149c4af48ead3d6650ee7ec5, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:21.314501 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4c620-b584-47b3-afa0-a65ab4316315 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:21.325215 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa029f3-d44e-4af8-93ae-bd8aa0e205d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2894018d1d984b839e5b900a1be2e896, user_id=bca0348930b94e289519b645b8ec8458, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:21.328060 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa029f3-d44e-4af8-93ae-bd8aa0e205d2 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:21.331151 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 791/1608] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:21 2022] PUT /identity/v3/projects/dcb15b699d5f4db990b0f9fd451eb470/users/33b8d13b149c4af48ead3d6650ee7ec5/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:21.331915 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4c620-b584-47b3-afa0-a65ab4316315 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:21.347194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 818/1609] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:21 2022] PUT /identity/v3/projects/2894018d1d984b839e5b900a1be2e896/users/bca0348930b94e289519b645b8ec8458/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:21.348551 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa029f3-d44e-4af8-93ae-bd8aa0e205d2 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:21.349409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:21.416122 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa029f3-d44e-4af8-93ae-bd8aa0e205d2 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:21.420531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2cee7-0d4a-4e8f-a50c-09f9c40d4b8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:21.421144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2cee7-0d4a-4e8f-a50c-09f9c40d4b8d service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:21.421430 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2cee7-0d4a-4e8f-a50c-09f9c40d4b8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:21.421706 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2cee7-0d4a-4e8f-a50c-09f9c40d4b8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:21.422246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4c620-b584-47b3-afa0-a65ab4316315 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:21.489212 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a919bc-2dd6-4343-a8db-c99d2d096574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:21.490103 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a919bc-2dd6-4343-a8db-c99d2d096574 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:21.490466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a919bc-2dd6-4343-a8db-c99d2d096574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:21.490806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a919bc-2dd6-4343-a8db-c99d2d096574 admin admin] PATH_INFO: `/v3/projects/078bf59c33cc43579aeb270262079a37/users/a3097d9c130746ca840bf85f33bc006b/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:21.509453 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a919bc-2dd6-4343-a8db-c99d2d096574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=078bf59c33cc43579aeb270262079a37, user_id=a3097d9c130746ca840bf85f33bc006b, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:21.511270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2cee7-0d4a-4e8f-a50c-09f9c40d4b8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:21.512728 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a919bc-2dd6-4343-a8db-c99d2d096574 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:21.512939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2cee7-0d4a-4e8f-a50c-09f9c40d4b8d service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:21.523694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 819/1610] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:21.525092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be2e79-bd6f-4f58-b91e-7a7c8579a85d None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:21.525473 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be2e79-bd6f-4f58-b91e-7a7c8579a85d None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:21.525860 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be2e79-bd6f-4f58-b91e-7a7c8579a85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:21.532895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 792/1611] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:21 2022] PUT /identity/v3/projects/078bf59c33cc43579aeb270262079a37/users/a3097d9c130746ca840bf85f33bc006b/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:21.532895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1226548-be82-4dbb-86fb-51de8a4492b5 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:21.532895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1226548-be82-4dbb-86fb-51de8a4492b5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:21.532895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1226548-be82-4dbb-86fb-51de8a4492b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:21.551563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-59be2e79-bd6f-4f58-b91e-7a7c8579a85d None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:21.552979 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-59be2e79-bd6f-4f58-b91e-7a7c8579a85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:21.564035 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d1226548-be82-4dbb-86fb-51de8a4492b5 None None] MFA Rules not processed for user `e1546ba5c727467c845925be6dbc4ec0`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:21.565254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d1226548-be82-4dbb-86fb-51de8a4492b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:21.629340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 820/1612] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:21.630242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be2e79-bd6f-4f58-b91e-7a7c8579a85d None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:21.640583 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be2e79-bd6f-4f58-b91e-7a7c8579a85d None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:21.645654 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a20bf-0e60-4dd1-87de-368fc8bb2c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:21.646308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a20bf-0e60-4dd1-87de-368fc8bb2c5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:21.646442 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 793/1613] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:21.647663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1c3a1-ea54-4499-9bf7-06e70fc85403 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:21.648770 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1c3a1-ea54-4499-9bf7-06e70fc85403 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:21.649222 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1c3a1-ea54-4499-9bf7-06e70fc85403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:21.650448 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a20bf-0e60-4dd1-87de-368fc8bb2c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:21.650854 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a20bf-0e60-4dd1-87de-368fc8bb2c5a admin admin] PATH_INFO: `/v3/projects/c4e0c1a9cc7547f79bedfc6ae85dae7f/users/1e31edc17c9240d88886adae1bdbae8a/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:21.669434 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a20bf-0e60-4dd1-87de-368fc8bb2c5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e0c1a9cc7547f79bedfc6ae85dae7f, user_id=1e31edc17c9240d88886adae1bdbae8a, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:21.672154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a20bf-0e60-4dd1-87de-368fc8bb2c5a admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:21.674893 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:21.691795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 821/1614] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:21 2022] PUT /identity/v3/projects/c4e0c1a9cc7547f79bedfc6ae85dae7f/users/1e31edc17c9240d88886adae1bdbae8a/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:21.691982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-3ad1c3a1-ea54-4499-9bf7-06e70fc85403 None None] MFA Rules not processed for user `33b8d13b149c4af48ead3d6650ee7ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:21.692085 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11858ea0-8b1d-4c39-84bc-227ab3543441 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:21.692085 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11858ea0-8b1d-4c39-84bc-227ab3543441 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:21.692085 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11858ea0-8b1d-4c39-84bc-227ab3543441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:21.692384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3ad1c3a1-ea54-4499-9bf7-06e70fc85403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:21.712308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:21.723037 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-11858ea0-8b1d-4c39-84bc-227ab3543441 None None] MFA Rules not processed for user `bca0348930b94e289519b645b8ec8458`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:21.724757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-11858ea0-8b1d-4c39-84bc-227ab3543441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:21.771673 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 794/1615] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:21 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:21.773159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0426e4c-64cb-47d9-860f-65cb8074b5c2 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:21.773531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0426e4c-64cb-47d9-860f-65cb8074b5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:21.773812 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0426e4c-64cb-47d9-860f-65cb8074b5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:21.811937 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 822/1616] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:21.813427 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11858ea0-8b1d-4c39-84bc-227ab3543441 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:21.817856 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e0426e4c-64cb-47d9-860f-65cb8074b5c2 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:21.821685 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-11858ea0-8b1d-4c39-84bc-227ab3543441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:21.822384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e0426e4c-64cb-47d9-860f-65cb8074b5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:21.958237 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 795/1617] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:21.959425 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0426e4c-64cb-47d9-860f-65cb8074b5c2 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:21.966287 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11858ea0-8b1d-4c39-84bc-227ab3543441 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:21.971289 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0426e4c-64cb-47d9-860f-65cb8074b5c2 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:21.971448 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ae6a6-eba5-4879-ad15-b961fb17e27c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:21.972058 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ae6a6-eba5-4879-ad15-b961fb17e27c service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:21.972394 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ae6a6-eba5-4879-ad15-b961fb17e27c service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:21.972675 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ae6a6-eba5-4879-ad15-b961fb17e27c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:21.976270 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b4542-90fa-452f-bce6-09cd77f779b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:21.976839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b4542-90fa-452f-bce6-09cd77f779b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:21.977112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b4542-90fa-452f-bce6-09cd77f779b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:21.977388 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b4542-90fa-452f-bce6-09cd77f779b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:21.978030 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b4542-90fa-452f-bce6-09cd77f779b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:21.980762 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b4542-90fa-452f-bce6-09cd77f779b4 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:21.984220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ae6a6-eba5-4879-ad15-b961fb17e27c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:21.986085 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ae6a6-eba5-4879-ad15-b961fb17e27c service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:21.998014 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 823/1618] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:21.999116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ae6a6-eba5-4879-ad15-b961fb17e27c service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:22.011201 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ae6a6-eba5-4879-ad15-b961fb17e27c service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:22.017021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69a0f2-dc2e-433d-b0d3-25efcd460d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:22.017629 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69a0f2-dc2e-433d-b0d3-25efcd460d77 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:22.017907 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69a0f2-dc2e-433d-b0d3-25efcd460d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:22.018178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69a0f2-dc2e-433d-b0d3-25efcd460d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:22.019107 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 796/1619] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:21 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:22.020128 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b4542-90fa-452f-bce6-09cd77f779b4 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:22.022008 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c69a0f2-dc2e-433d-b0d3-25efcd460d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:22.024080 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c69a0f2-dc2e-433d-b0d3-25efcd460d77 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:22.040682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 824/1620] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:22.040682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bca50b-b694-4bbf-b9ce-e337c8f4b8cc None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:22.040682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bca50b-b694-4bbf-b9ce-e337c8f4b8cc None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:22.040682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bca50b-b694-4bbf-b9ce-e337c8f4b8cc None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:22.042258 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 825/1621] 10.222.0.81 () {54 vars in 965 bytes} [Mon Sep 19 11:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:23:22.043191 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bca50b-b694-4bbf-b9ce-e337c8f4b8cc None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:22.109380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bca50b-b694-4bbf-b9ce-e337c8f4b8cc None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:22.113891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b4542-90fa-452f-bce6-09cd77f779b4 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:22.114319 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b1fbf-a135-4ba5-b5ca-4d0ac2b488fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:22.115115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b1fbf-a135-4ba5-b5ca-4d0ac2b488fd service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:22.115329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b1fbf-a135-4ba5-b5ca-4d0ac2b488fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:22.115532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b1fbf-a135-4ba5-b5ca-4d0ac2b488fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:22.120731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc87936-1d2e-4e4c-aaef-b24973c6150a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:22.121459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc87936-1d2e-4e4c-aaef-b24973c6150a admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:22.121812 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc87936-1d2e-4e4c-aaef-b24973c6150a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:22.122160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc87936-1d2e-4e4c-aaef-b24973c6150a admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:22.123142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc87936-1d2e-4e4c-aaef-b24973c6150a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:22.126399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc87936-1d2e-4e4c-aaef-b24973c6150a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:22.128360 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b1fbf-a135-4ba5-b5ca-4d0ac2b488fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:22.130485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b1fbf-a135-4ba5-b5ca-4d0ac2b488fd service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:22.142681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 826/1622] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:22.143716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b1fbf-a135-4ba5-b5ca-4d0ac2b488fd service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:22.157474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b1fbf-a135-4ba5-b5ca-4d0ac2b488fd service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:22.163503 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179576fe-4d27-4a5a-a1f2-4b38ac37f90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:22.164147 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179576fe-4d27-4a5a-a1f2-4b38ac37f90b admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:22.164426 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179576fe-4d27-4a5a-a1f2-4b38ac37f90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:22.164791 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179576fe-4d27-4a5a-a1f2-4b38ac37f90b admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:22.165430 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179576fe-4d27-4a5a-a1f2-4b38ac37f90b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:22.167602 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179576fe-4d27-4a5a-a1f2-4b38ac37f90b admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:22.169064 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 797/1623] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:22 2022] POST /identity/v3/projects => generated 357 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:22.170080 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc87936-1d2e-4e4c-aaef-b24973c6150a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:22.193484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:22.205567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 827/1624] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:22 2022] POST /identity/v3/users => generated 290 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:22.205567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d83ae-4cb4-4207-8bdc-4279f8234483 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:22.205567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d83ae-4cb4-4207-8bdc-4279f8234483 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:22.206489 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d83ae-4cb4-4207-8bdc-4279f8234483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:22.222764 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:22.286632 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc87936-1d2e-4e4c-aaef-b24973c6150a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:22.293312 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-6b2d83ae-4cb4-4207-8bdc-4279f8234483 None None] MFA Rules not processed for user `5abd70a18ccd407bafe3b61c8870986e`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:22.294706 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d686b05-7072-4a98-9a4c-60bfe39216b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:22.295713 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d686b05-7072-4a98-9a4c-60bfe39216b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:22.296120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d686b05-7072-4a98-9a4c-60bfe39216b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:22.296240 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-6b2d83ae-4cb4-4207-8bdc-4279f8234483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:22.296586 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d686b05-7072-4a98-9a4c-60bfe39216b3 admin admin] PATH_INFO: `/v3/projects/c4e0c1a9cc7547f79bedfc6ae85dae7f/users/1e31edc17c9240d88886adae1bdbae8a/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:22.299300 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d686b05-7072-4a98-9a4c-60bfe39216b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e0c1a9cc7547f79bedfc6ae85dae7f, user_id=1e31edc17c9240d88886adae1bdbae8a, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:22.303961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d686b05-7072-4a98-9a4c-60bfe39216b3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:22.321509 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 798/1625] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:22 2022] PUT /identity/v3/projects/c4e0c1a9cc7547f79bedfc6ae85dae7f/users/1e31edc17c9240d88886adae1bdbae8a/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:22.323851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d686b05-7072-4a98-9a4c-60bfe39216b3 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:22.452516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 828/1626] 10.222.0.81 () {58 vars in 1062 bytes} [Mon Sep 19 11:23:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:22.454410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d83ae-4cb4-4207-8bdc-4279f8234483 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:22.456893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d686b05-7072-4a98-9a4c-60bfe39216b3 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:22.463011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c565dde-3855-43b9-b0be-db613754522c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:22.464224 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c565dde-3855-43b9-b0be-db613754522c admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:22.464565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c565dde-3855-43b9-b0be-db613754522c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:22.464889 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c565dde-3855-43b9-b0be-db613754522c admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:22.465730 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c565dde-3855-43b9-b0be-db613754522c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:22.468031 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d83ae-4cb4-4207-8bdc-4279f8234483 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:22.468960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c565dde-3855-43b9-b0be-db613754522c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:22.474409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb75b9-fb77-415f-b450-e0df6e1883a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:22.475038 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb75b9-fb77-415f-b450-e0df6e1883a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:22.475517 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb75b9-fb77-415f-b450-e0df6e1883a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:22.475821 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb75b9-fb77-415f-b450-e0df6e1883a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:22.476800 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb75b9-fb77-415f-b450-e0df6e1883a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:22.478503 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb75b9-fb77-415f-b450-e0df6e1883a4 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:22.507391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 829/1627] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:22 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:22.508210 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c109448-d2fd-434a-b769-31fd8b4b5229 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:22.508350 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c109448-d2fd-434a-b769-31fd8b4b5229 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:22.512175 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c109448-d2fd-434a-b769-31fd8b4b5229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:22.536417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 799/1628] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:22 2022] POST /identity/v3/users => generated 285 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:22.538275 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c565dde-3855-43b9-b0be-db613754522c admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:22.584329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-3c109448-d2fd-434a-b769-31fd8b4b5229 None None] MFA Rules not processed for user `1e31edc17c9240d88886adae1bdbae8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:22.586668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3c109448-d2fd-434a-b769-31fd8b4b5229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:22.697003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c565dde-3855-43b9-b0be-db613754522c admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:22.705450 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c0de9-61ff-431f-a85b-230bd20e74b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:22.707117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c0de9-61ff-431f-a85b-230bd20e74b5 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:22.707117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c0de9-61ff-431f-a85b-230bd20e74b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:22.707117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c0de9-61ff-431f-a85b-230bd20e74b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:22.731280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 830/1629] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:22 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:22.732510 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c109448-d2fd-434a-b769-31fd8b4b5229 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:22.756199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c109448-d2fd-434a-b769-31fd8b4b5229 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:22.763180 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d90dd-7522-4e8b-a7ae-caaf88df1f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:22.763925 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d90dd-7522-4e8b-a7ae-caaf88df1f08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:22.764843 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d90dd-7522-4e8b-a7ae-caaf88df1f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:22.765227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d90dd-7522-4e8b-a7ae-caaf88df1f08 admin admin] PATH_INFO: `/v3/projects/193bbbe154d74a668ef975a9f2f4bace/users/4dc1e2ef45fb41379c3ca66f5f83bdd9/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:22.789423 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d90dd-7522-4e8b-a7ae-caaf88df1f08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=193bbbe154d74a668ef975a9f2f4bace, user_id=4dc1e2ef45fb41379c3ca66f5f83bdd9, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:22.793012 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d90dd-7522-4e8b-a7ae-caaf88df1f08 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:22.810851 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 831/1630] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:22 2022] PUT /identity/v3/projects/193bbbe154d74a668ef975a9f2f4bace/users/4dc1e2ef45fb41379c3ca66f5f83bdd9/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:22.813165 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d90dd-7522-4e8b-a7ae-caaf88df1f08 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:22.818124 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:22.846143 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c0de9-61ff-431f-a85b-230bd20e74b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:22.848171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c0de9-61ff-431f-a85b-230bd20e74b5 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:22.860010 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 800/1631] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:22.861169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c0de9-61ff-431f-a85b-230bd20e74b5 service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:22.862851 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:22.925928 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d90dd-7522-4e8b-a7ae-caaf88df1f08 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:22.933189 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9458ab-ebfb-455b-b80a-c59b8fe9fd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:22.934523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9458ab-ebfb-455b-b80a-c59b8fe9fd7f admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:22.934827 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9458ab-ebfb-455b-b80a-c59b8fe9fd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:22.935096 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9458ab-ebfb-455b-b80a-c59b8fe9fd7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:22.937006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9458ab-ebfb-455b-b80a-c59b8fe9fd7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:22.954131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c0de9-61ff-431f-a85b-230bd20e74b5 service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:22.958753 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9458ab-ebfb-455b-b80a-c59b8fe9fd7f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:22.984377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 832/1632] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:22 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:22.985911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9458ab-ebfb-455b-b80a-c59b8fe9fd7f admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:23.003164 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9458ab-ebfb-455b-b80a-c59b8fe9fd7f admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:23.011272 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948ea33-09b5-4d66-948b-4d4522c13e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:23.013081 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948ea33-09b5-4d66-948b-4d4522c13e94 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:23.013553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948ea33-09b5-4d66-948b-4d4522c13e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:23.014208 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948ea33-09b5-4d66-948b-4d4522c13e94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:23.015089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948ea33-09b5-4d66-948b-4d4522c13e94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:23.017616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948ea33-09b5-4d66-948b-4d4522c13e94 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:23.053409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 833/1633] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:22 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:23.053409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948ea33-09b5-4d66-948b-4d4522c13e94 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:23.082985 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948ea33-09b5-4d66-948b-4d4522c13e94 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:23.085731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc956b-0172-4b7d-9152-129080c42ab8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:23.086723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc956b-0172-4b7d-9152-129080c42ab8 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:23.088066 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc956b-0172-4b7d-9152-129080c42ab8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:23.089072 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc956b-0172-4b7d-9152-129080c42ab8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:23.098217 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff867c7-30e6-46e8-910d-c44b95c4aef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:23.099114 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff867c7-30e6-46e8-910d-c44b95c4aef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:23.099402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff867c7-30e6-46e8-910d-c44b95c4aef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:23.100189 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff867c7-30e6-46e8-910d-c44b95c4aef1 admin admin] PATH_INFO: `/v3/projects/dc45dc6bc8fd4939b8e7af39e1f40f8f/users/712a3d8e605c447d8d12c60489886304/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:23.131569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff867c7-30e6-46e8-910d-c44b95c4aef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc45dc6bc8fd4939b8e7af39e1f40f8f, user_id=712a3d8e605c447d8d12c60489886304, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:23.135213 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff867c7-30e6-46e8-910d-c44b95c4aef1 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:23.164519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 834/1634] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:23 2022] PUT /identity/v3/projects/dc45dc6bc8fd4939b8e7af39e1f40f8f/users/712a3d8e605c447d8d12c60489886304/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:23.165768 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff867c7-30e6-46e8-910d-c44b95c4aef1 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:23.211041 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc956b-0172-4b7d-9152-129080c42ab8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:23.216491 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc956b-0172-4b7d-9152-129080c42ab8 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:23.240124 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 801/1635] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:22 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:23.241186 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc956b-0172-4b7d-9152-129080c42ab8 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:23.268944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc956b-0172-4b7d-9152-129080c42ab8 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:23.339918 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff867c7-30e6-46e8-910d-c44b95c4aef1 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:23.357386 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226d213-93cc-4711-9875-bcb991d35183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:23.358177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226d213-93cc-4711-9875-bcb991d35183 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:23.358495 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226d213-93cc-4711-9875-bcb991d35183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:23.358773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226d213-93cc-4711-9875-bcb991d35183 admin admin] PATH_INFO: `/v3/projects/193bbbe154d74a668ef975a9f2f4bace/users/4dc1e2ef45fb41379c3ca66f5f83bdd9/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:23.361682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226d213-93cc-4711-9875-bcb991d35183 admin admin] RBAC: Authorizing `identity:create_grant(project_id=193bbbe154d74a668ef975a9f2f4bace, user_id=4dc1e2ef45fb41379c3ca66f5f83bdd9, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:23.364971 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226d213-93cc-4711-9875-bcb991d35183 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:23.376712 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:23.401663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f57ed8-acf8-430a-8f2e-db2163afb824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:23.401926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 835/1636] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:23 2022] PUT /identity/v3/projects/193bbbe154d74a668ef975a9f2f4bace/users/4dc1e2ef45fb41379c3ca66f5f83bdd9/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:23.404203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f57ed8-acf8-430a-8f2e-db2163afb824 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:23.405882 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f57ed8-acf8-430a-8f2e-db2163afb824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:23.406386 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f57ed8-acf8-430a-8f2e-db2163afb824 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:23.407417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f57ed8-acf8-430a-8f2e-db2163afb824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:23.411475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f57ed8-acf8-430a-8f2e-db2163afb824 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:23.419685 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba8609-f7fc-452d-8803-a0928d4fd286 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:23.419995 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba8609-f7fc-452d-8803-a0928d4fd286 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:23.420260 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba8609-f7fc-452d-8803-a0928d4fd286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:23.437933 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 802/1637] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:23.441936 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f57ed8-acf8-430a-8f2e-db2163afb824 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:23.477452 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f57ed8-acf8-430a-8f2e-db2163afb824 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:23.498065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-45ba8609-f7fc-452d-8803-a0928d4fd286 None None] MFA Rules not processed for user `4dc1e2ef45fb41379c3ca66f5f83bdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:23.498065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-45ba8609-f7fc-452d-8803-a0928d4fd286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:23.645074 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d8da3-4253-435b-84ef-cfb9b125e3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:23.645761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d8da3-4253-435b-84ef-cfb9b125e3d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:23.646063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d8da3-4253-435b-84ef-cfb9b125e3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:23.646304 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d8da3-4253-435b-84ef-cfb9b125e3d5 admin admin] PATH_INFO: `/v3/projects/dc45dc6bc8fd4939b8e7af39e1f40f8f/users/712a3d8e605c447d8d12c60489886304/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:23.649149 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d8da3-4253-435b-84ef-cfb9b125e3d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc45dc6bc8fd4939b8e7af39e1f40f8f, user_id=712a3d8e605c447d8d12c60489886304, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:23.661550 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d8da3-4253-435b-84ef-cfb9b125e3d5 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:23.682604 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 836/1638] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:23.684664 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 803/1639] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:23 2022] PUT /identity/v3/projects/dc45dc6bc8fd4939b8e7af39e1f40f8f/users/712a3d8e605c447d8d12c60489886304/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:23.685258 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba8609-f7fc-452d-8803-a0928d4fd286 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:23.686732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5d4cc-5b3c-4fe8-8204-667f586cb930 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:23.687336 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5d4cc-5b3c-4fe8-8204-667f586cb930 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:23.687852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5d4cc-5b3c-4fe8-8204-667f586cb930 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:23.690638 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 804/1640] 10.222.0.81 () {54 vars in 965 bytes} [Mon Sep 19 11:23:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:23:23.698649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf7b57-c710-485c-b7cc-ba47d3d7810e None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:23.698937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf7b57-c710-485c-b7cc-ba47d3d7810e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:23.700086 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf7b57-c710-485c-b7cc-ba47d3d7810e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:23.712886 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba8609-f7fc-452d-8803-a0928d4fd286 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:23.724917 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:23.756453 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-cfdf7b57-c710-485c-b7cc-ba47d3d7810e None None] MFA Rules not processed for user `712a3d8e605c447d8d12c60489886304`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:23.759234 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cfdf7b57-c710-485c-b7cc-ba47d3d7810e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:23.861711 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1aa66-6f4d-4ddd-8ec3-f2ab4923834e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:23.862365 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1aa66-6f4d-4ddd-8ec3-f2ab4923834e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:23.862713 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1aa66-6f4d-4ddd-8ec3-f2ab4923834e admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:23.863027 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1aa66-6f4d-4ddd-8ec3-f2ab4923834e admin admin] PATH_INFO: `/v3/users/53d79984cadc4fb2a477e05c5c7a7f11` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:23.864215 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1aa66-6f4d-4ddd-8ec3-f2ab4923834e admin admin] RBAC: Authorizing `identity:delete_user(user_id=53d79984cadc4fb2a477e05c5c7a7f11)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:23.866584 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1aa66-6f4d-4ddd-8ec3-f2ab4923834e admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:23.931243 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 805/1641] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:23 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:23:23.931243 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bf5a5-8d0e-4638-8d92-302bb30b65c1 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:23.931243 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bf5a5-8d0e-4638-8d92-302bb30b65c1 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:23.931243 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bf5a5-8d0e-4638-8d92-302bb30b65c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:23.970181 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-084bf5a5-8d0e-4638-8d92-302bb30b65c1 None None] MFA Rules not processed for user `5abd70a18ccd407bafe3b61c8870986e`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:23.970535 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-0fd1aa66-6f4d-4ddd-8ec3-f2ab4923834e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53d79984cadc4fb2a477e05c5c7a7f11'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:23.971911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-0fd1aa66-6f4d-4ddd-8ec3-f2ab4923834e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53d79984cadc4fb2a477e05c5c7a7f11'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:23.973099 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-0fd1aa66-6f4d-4ddd-8ec3-f2ab4923834e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53d79984cadc4fb2a477e05c5c7a7f11'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:23.973972 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-084bf5a5-8d0e-4638-8d92-302bb30b65c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:23.991349 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-0fd1aa66-6f4d-4ddd-8ec3-f2ab4923834e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53d79984cadc4fb2a477e05c5c7a7f11'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:24.019976 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-0fd1aa66-6f4d-4ddd-8ec3-f2ab4923834e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53d79984cadc4fb2a477e05c5c7a7f11'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:24.049624 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 837/1642] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:23:23 2022] DELETE /identity/v3/users/53d79984cadc4fb2a477e05c5c7a7f11 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:24.051209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1aa66-6f4d-4ddd-8ec3-f2ab4923834e admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:24.055733 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-0fd1aa66-6f4d-4ddd-8ec3-f2ab4923834e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:24.120863 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:24.139757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1aa66-6f4d-4ddd-8ec3-f2ab4923834e admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:24.147883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 806/1643] 10.222.0.81 () {58 vars in 1062 bytes} [Mon Sep 19 11:23:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:24.152490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bf5a5-8d0e-4638-8d92-302bb30b65c1 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:24.156989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-084bf5a5-8d0e-4638-8d92-302bb30b65c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:24.160697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e5347-58a7-4aa3-8fe9-06cdd5ebd56c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:24.161825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e5347-58a7-4aa3-8fe9-06cdd5ebd56c service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:24.161825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e5347-58a7-4aa3-8fe9-06cdd5ebd56c service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:24.162147 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e5347-58a7-4aa3-8fe9-06cdd5ebd56c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:24.166161 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-7d5e5347-58a7-4aa3-8fe9-06cdd5ebd56c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:24.251902 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:24.267916 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bf5a5-8d0e-4638-8d92-302bb30b65c1 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:24.273488 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa322632-39b0-4d93-9576-0a4f08c49a2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:24.274298 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa322632-39b0-4d93-9576-0a4f08c49a2f service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:24.274583 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa322632-39b0-4d93-9576-0a4f08c49a2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:24.274857 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa322632-39b0-4d93-9576-0a4f08c49a2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:24.276781 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-aa322632-39b0-4d93-9576-0a4f08c49a2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:24.316562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e5347-58a7-4aa3-8fe9-06cdd5ebd56c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:24.319911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e5347-58a7-4aa3-8fe9-06cdd5ebd56c service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:24.352916 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 838/1644] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:24 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:24.358816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747f32c-04a6-4c1f-a1ad-feb3b6c5696d None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:24.359320 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747f32c-04a6-4c1f-a1ad-feb3b6c5696d None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:24.360056 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747f32c-04a6-4c1f-a1ad-feb3b6c5696d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:24.410548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-d747f32c-04a6-4c1f-a1ad-feb3b6c5696d None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:24.416470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-d747f32c-04a6-4c1f-a1ad-feb3b6c5696d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:24.416722 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa322632-39b0-4d93-9576-0a4f08c49a2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:24.421213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa322632-39b0-4d93-9576-0a4f08c49a2f service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:24.464315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 807/1645] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:24 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:24.465658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1504d5-e647-4648-9bc6-72466a9a8f6f None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:24.465940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1504d5-e647-4648-9bc6-72466a9a8f6f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:24.466226 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1504d5-e647-4648-9bc6-72466a9a8f6f None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:24.471270 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 808/1646] 10.222.0.81 () {54 vars in 912 bytes} [Mon Sep 19 11:23:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:23:24.482795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b719a-238b-4265-b3af-8ee7de203539 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:24.483193 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b719a-238b-4265-b3af-8ee7de203539 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:24.483367 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b719a-238b-4265-b3af-8ee7de203539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:24.523221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c59b719a-238b-4265-b3af-8ee7de203539 None None] MFA Rules not processed for user `5abd70a18ccd407bafe3b61c8870986e`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:24.524506 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c59b719a-238b-4265-b3af-8ee7de203539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:24.610825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 839/1647] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:24.625829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747f32c-04a6-4c1f-a1ad-feb3b6c5696d None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:24.627288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-d747f32c-04a6-4c1f-a1ad-feb3b6c5696d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:24.724911 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 809/1648] 10.222.0.81 () {58 vars in 1009 bytes} [Mon Sep 19 11:23:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:23:24.764564 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747f32c-04a6-4c1f-a1ad-feb3b6c5696d None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:24.764918 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b719a-238b-4265-b3af-8ee7de203539 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:24.773812 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e38092-3dde-43cb-84bd-271bda441701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:24.774455 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e38092-3dde-43cb-84bd-271bda441701 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:24.774744 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e38092-3dde-43cb-84bd-271bda441701 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:24.775098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e38092-3dde-43cb-84bd-271bda441701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:24.779889 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b719a-238b-4265-b3af-8ee7de203539 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:24.790173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec9328-430b-4900-984e-5911cfe6df16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:24.791091 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec9328-430b-4900-984e-5911cfe6df16 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:24.793716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e38092-3dde-43cb-84bd-271bda441701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:24.795156 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e38092-3dde-43cb-84bd-271bda441701 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:24.797289 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec9328-430b-4900-984e-5911cfe6df16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:24.797593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec9328-430b-4900-984e-5911cfe6df16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:24.823553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 840/1649] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:23:24.824441 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e38092-3dde-43cb-84bd-271bda441701 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:24.827606 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec9328-430b-4900-984e-5911cfe6df16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:24.829781 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec9328-430b-4900-984e-5911cfe6df16 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:24.845922 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e38092-3dde-43cb-84bd-271bda441701 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:24.859576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d40dc8-5af9-492b-a335-40a903935c2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:24.860227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d40dc8-5af9-492b-a335-40a903935c2b service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:24.860510 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d40dc8-5af9-492b-a335-40a903935c2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:24.860787 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d40dc8-5af9-492b-a335-40a903935c2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:24.861461 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 810/1650] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:24.862630 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a2d40dc8-5af9-492b-a335-40a903935c2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:24.987012 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d40dc8-5af9-492b-a335-40a903935c2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:24.988498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d40dc8-5af9-492b-a335-40a903935c2b service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:25.013102 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 841/1651] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:24 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:25.073626 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4190430-6ae2-4c3f-ab30-aaec6928ee1c None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:25.073910 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4190430-6ae2-4c3f-ab30-aaec6928ee1c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:25.074178 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4190430-6ae2-4c3f-ab30-aaec6928ee1c None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:25.076877 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 811/1652] 10.222.0.81 () {54 vars in 912 bytes} [Mon Sep 19 11:23:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:23:25.082616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdd160-bb5b-4c3e-b61a-9ab1a6cd99c8 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:25.083098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdd160-bb5b-4c3e-b61a-9ab1a6cd99c8 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:25.083575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdd160-bb5b-4c3e-b61a-9ab1a6cd99c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:25.138904 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-edcdd160-bb5b-4c3e-b61a-9ab1a6cd99c8 None None] MFA Rules not processed for user `5abd70a18ccd407bafe3b61c8870986e`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:25.145538 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-edcdd160-bb5b-4c3e-b61a-9ab1a6cd99c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:25.196059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 842/1653] 10.222.0.81 () {58 vars in 1009 bytes} [Mon Sep 19 11:23:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:25.261536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4190430-6ae2-4c3f-ab30-aaec6928ee1c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:25.283552 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4190430-6ae2-4c3f-ab30-aaec6928ee1c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:25.291790 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591af20-8eb6-4811-bb14-f274bc8eec28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:25.292828 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591af20-8eb6-4811-bb14-f274bc8eec28 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:25.293206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591af20-8eb6-4811-bb14-f274bc8eec28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:25.293550 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591af20-8eb6-4811-bb14-f274bc8eec28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:25.326393 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591af20-8eb6-4811-bb14-f274bc8eec28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:25.342288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591af20-8eb6-4811-bb14-f274bc8eec28 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:25.362736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 812/1654] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:25.577715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcdd160-bb5b-4c3e-b61a-9ab1a6cd99c8 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:25.577715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-edcdd160-bb5b-4c3e-b61a-9ab1a6cd99c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:25.615588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcdd160-bb5b-4c3e-b61a-9ab1a6cd99c8 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:25.633396 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518f434-f50d-4e9c-94e6-dd2028511991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:25.634453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518f434-f50d-4e9c-94e6-dd2028511991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:25.634786 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518f434-f50d-4e9c-94e6-dd2028511991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:25.635320 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518f434-f50d-4e9c-94e6-dd2028511991 admin admin] PATH_INFO: `/v3/users/bca0348930b94e289519b645b8ec8458` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:25.637004 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518f434-f50d-4e9c-94e6-dd2028511991 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bca0348930b94e289519b645b8ec8458)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:25.639012 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518f434-f50d-4e9c-94e6-dd2028511991 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:25.648735 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:25.760939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-1518f434-f50d-4e9c-94e6-dd2028511991 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bca0348930b94e289519b645b8ec8458'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:25.762960 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-1518f434-f50d-4e9c-94e6-dd2028511991 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bca0348930b94e289519b645b8ec8458'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:25.763847 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-1518f434-f50d-4e9c-94e6-dd2028511991 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bca0348930b94e289519b645b8ec8458'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:25.787849 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-1518f434-f50d-4e9c-94e6-dd2028511991 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bca0348930b94e289519b645b8ec8458'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:25.833583 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-1518f434-f50d-4e9c-94e6-dd2028511991 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bca0348930b94e289519b645b8ec8458'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:25.863800 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 843/1655] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:23:25 2022] DELETE /identity/v3/users/bca0348930b94e289519b645b8ec8458 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:25.868092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f8472-02fa-4e98-b427-8b406c3097d4 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:25.868393 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f8472-02fa-4e98-b427-8b406c3097d4 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:25.869305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f8472-02fa-4e98-b427-8b406c3097d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:25.918735 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9d1f8472-02fa-4e98-b427-8b406c3097d4 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:25.923415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9d1f8472-02fa-4e98-b427-8b406c3097d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:25.941170 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:26.050960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 813/1656] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:26.092095 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518f434-f50d-4e9c-94e6-dd2028511991 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:26.093711 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-1518f434-f50d-4e9c-94e6-dd2028511991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:26.284827 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518f434-f50d-4e9c-94e6-dd2028511991 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:26.291066 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59aae68-392d-46b8-ab35-459e4703e632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:26.292142 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aae68-392d-46b8-ab35-459e4703e632 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:26.303063 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aae68-392d-46b8-ab35-459e4703e632 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:26.303487 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aae68-392d-46b8-ab35-459e4703e632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:26.317906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59aae68-392d-46b8-ab35-459e4703e632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:26.319822 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59aae68-392d-46b8-ab35-459e4703e632 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:26.333131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 844/1657] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:26.695809 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f8472-02fa-4e98-b427-8b406c3097d4 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:26.697341 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9d1f8472-02fa-4e98-b427-8b406c3097d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:26.724221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f8472-02fa-4e98-b427-8b406c3097d4 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:26.735713 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad144331-793a-4272-8ab6-03f5d3704c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:26.736649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad144331-793a-4272-8ab6-03f5d3704c06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:26.737011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad144331-793a-4272-8ab6-03f5d3704c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:26.737423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad144331-793a-4272-8ab6-03f5d3704c06 admin admin] PATH_INFO: `/v3/projects/cfccfb243e074da4958a5c4855232cf1` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:26.738914 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad144331-793a-4272-8ab6-03f5d3704c06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfccfb243e074da4958a5c4855232cf1)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:26.746599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad144331-793a-4272-8ab6-03f5d3704c06 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:26.889162 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ad144331-793a-4272-8ab6-03f5d3704c06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfccfb243e074da4958a5c4855232cf1', 'request_id': 'req-ad144331-793a-4272-8ab6-03f5d3704c06'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:26.909861 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ad144331-793a-4272-8ab6-03f5d3704c06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfccfb243e074da4958a5c4855232cf1', 'request_id': 'req-ad144331-793a-4272-8ab6-03f5d3704c06'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:26.925508 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ad144331-793a-4272-8ab6-03f5d3704c06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfccfb243e074da4958a5c4855232cf1', 'request_id': 'req-ad144331-793a-4272-8ab6-03f5d3704c06'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:26.961233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ad144331-793a-4272-8ab6-03f5d3704c06 admin admin] The token cache is being invalidate because project cfccfb243e074da4958a5c4855232cf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:23:26.964222 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ad144331-793a-4272-8ab6-03f5d3704c06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:26.966523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ad144331-793a-4272-8ab6-03f5d3704c06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:26.975160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 814/1658] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:23:26 2022] DELETE /identity/v3/projects/cfccfb243e074da4958a5c4855232cf1 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:26.978907 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59aae68-392d-46b8-ab35-459e4703e632 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:26.994726 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59aae68-392d-46b8-ab35-459e4703e632 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:27.267186 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd8aba-ce8a-4d74-b8c7-b8f51fadd759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:27.267667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd8aba-ce8a-4d74-b8c7-b8f51fadd759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:27.269131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd8aba-ce8a-4d74-b8c7-b8f51fadd759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:27.269424 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd8aba-ce8a-4d74-b8c7-b8f51fadd759 admin admin] PATH_INFO: `/v3/users/1e31edc17c9240d88886adae1bdbae8a` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:27.272613 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd8aba-ce8a-4d74-b8c7-b8f51fadd759 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e31edc17c9240d88886adae1bdbae8a)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:27.277096 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd8aba-ce8a-4d74-b8c7-b8f51fadd759 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:27.353925 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:27.427972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e9cd8aba-ce8a-4d74-b8c7-b8f51fadd759 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e31edc17c9240d88886adae1bdbae8a'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:27.432797 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e9cd8aba-ce8a-4d74-b8c7-b8f51fadd759 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e31edc17c9240d88886adae1bdbae8a'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:27.439739 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e9cd8aba-ce8a-4d74-b8c7-b8f51fadd759 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e31edc17c9240d88886adae1bdbae8a'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:27.461872 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e9cd8aba-ce8a-4d74-b8c7-b8f51fadd759 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e31edc17c9240d88886adae1bdbae8a'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:27.502923 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e9cd8aba-ce8a-4d74-b8c7-b8f51fadd759 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e31edc17c9240d88886adae1bdbae8a'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:27.532498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 845/1659] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:23:26 2022] DELETE /identity/v3/users/1e31edc17c9240d88886adae1bdbae8a => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:28.360056 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad144331-793a-4272-8ab6-03f5d3704c06 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:28.362746 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ad144331-793a-4272-8ab6-03f5d3704c06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:28.390046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad144331-793a-4272-8ab6-03f5d3704c06 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:28.529725 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e656e5e-37c9-48f9-a049-864f960ba894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:28.530678 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e656e5e-37c9-48f9-a049-864f960ba894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:28.531223 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e656e5e-37c9-48f9-a049-864f960ba894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:28.532321 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e656e5e-37c9-48f9-a049-864f960ba894 admin admin] PATH_INFO: `/v3/users/e1546ba5c727467c845925be6dbc4ec0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:28.534864 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e656e5e-37c9-48f9-a049-864f960ba894 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1546ba5c727467c845925be6dbc4ec0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:28.536844 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: WARNING py.warnings [None req-5e656e5e-37c9-48f9-a049-864f960ba894 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:23:28.536844 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: warnings.warn(msg) Sep 19 11:23:28.536844 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: Sep 19 11:23:28.539881 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e656e5e-37c9-48f9-a049-864f960ba894 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:28.638775 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd8aba-ce8a-4d74-b8c7-b8f51fadd759 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:28.644117 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e9cd8aba-ce8a-4d74-b8c7-b8f51fadd759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:28.661539 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:28.680635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd8aba-ce8a-4d74-b8c7-b8f51fadd759 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:28.696097 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989243a-3fec-44a6-9291-33fd8fde308d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:28.701011 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989243a-3fec-44a6-9291-33fd8fde308d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:28.701376 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989243a-3fec-44a6-9291-33fd8fde308d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:28.701661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989243a-3fec-44a6-9291-33fd8fde308d admin admin] PATH_INFO: `/v3/projects/2894018d1d984b839e5b900a1be2e896` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:28.709762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989243a-3fec-44a6-9291-33fd8fde308d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2894018d1d984b839e5b900a1be2e896)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:28.713501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: WARNING py.warnings [None req-e989243a-3fec-44a6-9291-33fd8fde308d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 11:23:28.713501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: warnings.warn(msg) Sep 19 11:23:28.713501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: Sep 19 11:23:28.713739 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5e656e5e-37c9-48f9-a049-864f960ba894 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1546ba5c727467c845925be6dbc4ec0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:28.723044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989243a-3fec-44a6-9291-33fd8fde308d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:28.745538 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5e656e5e-37c9-48f9-a049-864f960ba894 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1546ba5c727467c845925be6dbc4ec0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:28.746587 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5e656e5e-37c9-48f9-a049-864f960ba894 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1546ba5c727467c845925be6dbc4ec0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:28.747481 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5e656e5e-37c9-48f9-a049-864f960ba894 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1546ba5c727467c845925be6dbc4ec0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:28.814535 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5e656e5e-37c9-48f9-a049-864f960ba894 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1546ba5c727467c845925be6dbc4ec0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:28.849653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 815/1660] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:23:28 2022] DELETE /identity/v3/users/e1546ba5c727467c845925be6dbc4ec0 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:28.860540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13adfd7-9d5e-415d-9aa2-b3eabc32e5b6 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:28.861134 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13adfd7-9d5e-415d-9aa2-b3eabc32e5b6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:28.867620 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13adfd7-9d5e-415d-9aa2-b3eabc32e5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:28.894902 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e989243a-3fec-44a6-9291-33fd8fde308d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2894018d1d984b839e5b900a1be2e896', 'request_id': 'req-e989243a-3fec-44a6-9291-33fd8fde308d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:28.927157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e989243a-3fec-44a6-9291-33fd8fde308d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2894018d1d984b839e5b900a1be2e896', 'request_id': 'req-e989243a-3fec-44a6-9291-33fd8fde308d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:28.929672 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d13adfd7-9d5e-415d-9aa2-b3eabc32e5b6 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:28.932516 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d13adfd7-9d5e-415d-9aa2-b3eabc32e5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:28.955488 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e989243a-3fec-44a6-9291-33fd8fde308d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2894018d1d984b839e5b900a1be2e896', 'request_id': 'req-e989243a-3fec-44a6-9291-33fd8fde308d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:28.997319 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e989243a-3fec-44a6-9291-33fd8fde308d admin admin] The token cache is being invalidate because project 2894018d1d984b839e5b900a1be2e896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:23:28.997319 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e989243a-3fec-44a6-9291-33fd8fde308d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:29.000317 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e989243a-3fec-44a6-9291-33fd8fde308d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:29.003948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 846/1661] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:23:28 2022] DELETE /identity/v3/projects/2894018d1d984b839e5b900a1be2e896 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:23:29.009136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aaffed-d4f0-4bbd-9f0a-5bb03ce73fdd None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:29.009434 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aaffed-d4f0-4bbd-9f0a-5bb03ce73fdd None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:29.009710 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aaffed-d4f0-4bbd-9f0a-5bb03ce73fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:29.069160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-a1aaffed-d4f0-4bbd-9f0a-5bb03ce73fdd None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:29.071735 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a1aaffed-d4f0-4bbd-9f0a-5bb03ce73fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:29.186145 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 816/1662] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:23:29.251857 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 847/1663] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:29.256208 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283f626-dcd9-42c8-be1f-ff1456101496 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:29.256704 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283f626-dcd9-42c8-be1f-ff1456101496 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:29.257199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283f626-dcd9-42c8-be1f-ff1456101496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:29.333008 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d283f626-dcd9-42c8-be1f-ff1456101496 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:29.334540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d283f626-dcd9-42c8-be1f-ff1456101496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:29.368847 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aaffed-d4f0-4bbd-9f0a-5bb03ce73fdd None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:29.369349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 817/1664] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:29.373794 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a1aaffed-d4f0-4bbd-9f0a-5bb03ce73fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:29.377084 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283f626-dcd9-42c8-be1f-ff1456101496 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:29.410486 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283f626-dcd9-42c8-be1f-ff1456101496 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:29.434053 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6bd51-ecf9-45b3-9cc9-ad2f456ab150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:29.434688 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6bd51-ecf9-45b3-9cc9-ad2f456ab150 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:29.434969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6bd51-ecf9-45b3-9cc9-ad2f456ab150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:29.435242 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6bd51-ecf9-45b3-9cc9-ad2f456ab150 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:29.435934 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6bd51-ecf9-45b3-9cc9-ad2f456ab150 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:23:29.436236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6bd51-ecf9-45b3-9cc9-ad2f456ab150 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:29.438832 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6bd51-ecf9-45b3-9cc9-ad2f456ab150 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:29.506912 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 818/1665] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:23:29.509178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:29.511669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6bd51-ecf9-45b3-9cc9-ad2f456ab150 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:29.549598 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6bd51-ecf9-45b3-9cc9-ad2f456ab150 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:29.582008 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4941f9f-e997-4b8a-ae68-d49def5b3a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:29.590674 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4941f9f-e997-4b8a-ae68-d49def5b3a0f admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:29.590962 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4941f9f-e997-4b8a-ae68-d49def5b3a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:29.591239 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4941f9f-e997-4b8a-ae68-d49def5b3a0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:29.592009 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4941f9f-e997-4b8a-ae68-d49def5b3a0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:29.593958 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4941f9f-e997-4b8a-ae68-d49def5b3a0f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:29.631074 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aaffed-d4f0-4bbd-9f0a-5bb03ce73fdd None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:29.635071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a8214-64f4-4dbb-8152-2a266bd4d3b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:29.636310 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a8214-64f4-4dbb-8152-2a266bd4d3b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:29.636587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a8214-64f4-4dbb-8152-2a266bd4d3b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:29.636855 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a8214-64f4-4dbb-8152-2a266bd4d3b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:29.667249 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a8214-64f4-4dbb-8152-2a266bd4d3b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:29.675463 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a8214-64f4-4dbb-8152-2a266bd4d3b0 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:29.675589 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 819/1666] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:29 2022] POST /identity/v3/projects => generated 405 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:23:29.679623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4941f9f-e997-4b8a-ae68-d49def5b3a0f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:29.697358 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 848/1667] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:29.847574 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4941f9f-e997-4b8a-ae68-d49def5b3a0f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:29.877831 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddb867-bb90-4efa-b935-b8f4058aad23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:29.878391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddb867-bb90-4efa-b935-b8f4058aad23 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:29.878678 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddb867-bb90-4efa-b935-b8f4058aad23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:29.878956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddb867-bb90-4efa-b935-b8f4058aad23 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:29.879615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ddb867-bb90-4efa-b935-b8f4058aad23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:29.881923 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ddb867-bb90-4efa-b935-b8f4058aad23 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:29.929669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 820/1668] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:29 2022] POST /identity/v3/users => generated 303 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:23:29.931143 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a8214-64f4-4dbb-8152-2a266bd4d3b0 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:29.952408 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a8214-64f4-4dbb-8152-2a266bd4d3b0 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:29.959228 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e6edd-8977-40d7-8d9b-8e36511de17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:29.960249 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e6edd-8977-40d7-8d9b-8e36511de17f admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:29.960530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e6edd-8977-40d7-8d9b-8e36511de17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:29.960802 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e6edd-8977-40d7-8d9b-8e36511de17f admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:29.961457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e6edd-8977-40d7-8d9b-8e36511de17f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:29.963184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e6edd-8977-40d7-8d9b-8e36511de17f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:29.982631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 849/1669] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:29.983888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddb867-bb90-4efa-b935-b8f4058aad23 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:29.989257 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:29.998295 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddb867-bb90-4efa-b935-b8f4058aad23 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:30.008598 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2231bab-db41-4565-94e5-6f78fe3ac352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:30.009235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2231bab-db41-4565-94e5-6f78fe3ac352 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:30.009521 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2231bab-db41-4565-94e5-6f78fe3ac352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:30.009796 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2231bab-db41-4565-94e5-6f78fe3ac352 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:30.010487 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2231bab-db41-4565-94e5-6f78fe3ac352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:30.012546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2231bab-db41-4565-94e5-6f78fe3ac352 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:30.028842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 821/1670] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:30.030298 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e6edd-8977-40d7-8d9b-8e36511de17f admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:30.051433 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e6edd-8977-40d7-8d9b-8e36511de17f admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:30.066822 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573273e7-c039-41b6-a348-038902fa7e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:30.067911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573273e7-c039-41b6-a348-038902fa7e95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:30.068269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573273e7-c039-41b6-a348-038902fa7e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:30.068936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573273e7-c039-41b6-a348-038902fa7e95 admin admin] PATH_INFO: `/v3/projects/7c8ef32903e844e0ba16a7cc9cf9d608/users/07cc7bb8955e4255a2a9d9d717f06413/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:30.129679 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573273e7-c039-41b6-a348-038902fa7e95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c8ef32903e844e0ba16a7cc9cf9d608, user_id=07cc7bb8955e4255a2a9d9d717f06413, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:30.140236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573273e7-c039-41b6-a348-038902fa7e95 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:30.178643 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 850/1671] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:30 2022] PUT /identity/v3/projects/7c8ef32903e844e0ba16a7cc9cf9d608/users/07cc7bb8955e4255a2a9d9d717f06413/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:30.182548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5bdcdf-c857-4fa5-acbb-371dc61e45d3 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:30.183246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5bdcdf-c857-4fa5-acbb-371dc61e45d3 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:30.183507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5bdcdf-c857-4fa5-acbb-371dc61e45d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:30.220619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8e5bdcdf-c857-4fa5-acbb-371dc61e45d3 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:30.221851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8e5bdcdf-c857-4fa5-acbb-371dc61e45d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:30.352820 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573273e7-c039-41b6-a348-038902fa7e95 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:30.355240 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-573273e7-c039-41b6-a348-038902fa7e95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:30.381104 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 822/1672] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:30.399213 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573273e7-c039-41b6-a348-038902fa7e95 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:30.415709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6f87a-05bb-45e9-8e51-2167e0c48495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:30.416433 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6f87a-05bb-45e9-8e51-2167e0c48495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:30.416808 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6f87a-05bb-45e9-8e51-2167e0c48495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:30.417082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6f87a-05bb-45e9-8e51-2167e0c48495 admin admin] PATH_INFO: `/v3/projects/c4e0c1a9cc7547f79bedfc6ae85dae7f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:30.422670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6f87a-05bb-45e9-8e51-2167e0c48495 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4e0c1a9cc7547f79bedfc6ae85dae7f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:30.424744 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6f87a-05bb-45e9-8e51-2167e0c48495 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:30.433863 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5bdcdf-c857-4fa5-acbb-371dc61e45d3 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:30.435715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8e5bdcdf-c857-4fa5-acbb-371dc61e45d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:30.607245 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5cb6f87a-05bb-45e9-8e51-2167e0c48495 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4e0c1a9cc7547f79bedfc6ae85dae7f', 'request_id': 'req-5cb6f87a-05bb-45e9-8e51-2167e0c48495'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:30.627349 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5cb6f87a-05bb-45e9-8e51-2167e0c48495 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4e0c1a9cc7547f79bedfc6ae85dae7f', 'request_id': 'req-5cb6f87a-05bb-45e9-8e51-2167e0c48495'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:30.630189 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5bdcdf-c857-4fa5-acbb-371dc61e45d3 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:30.641365 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5cb6f87a-05bb-45e9-8e51-2167e0c48495 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4e0c1a9cc7547f79bedfc6ae85dae7f', 'request_id': 'req-5cb6f87a-05bb-45e9-8e51-2167e0c48495'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:30.669353 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5cb6f87a-05bb-45e9-8e51-2167e0c48495 admin admin] The token cache is being invalidate because project c4e0c1a9cc7547f79bedfc6ae85dae7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:23:30.669979 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5cb6f87a-05bb-45e9-8e51-2167e0c48495 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:30.671207 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5cb6f87a-05bb-45e9-8e51-2167e0c48495 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:30.675429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 851/1673] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:23:30 2022] DELETE /identity/v3/projects/c4e0c1a9cc7547f79bedfc6ae85dae7f => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:23:30.686099 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5c149-6e95-484d-a9df-405bb2a0d453 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:30.687732 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5c149-6e95-484d-a9df-405bb2a0d453 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:30.688335 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5c149-6e95-484d-a9df-405bb2a0d453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:30.756112 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-b3f5c149-6e95-484d-a9df-405bb2a0d453 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:30.759621 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b3f5c149-6e95-484d-a9df-405bb2a0d453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:30.786292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb38d47-8e45-4187-a4f6-5617b33fd734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:30.789298 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb38d47-8e45-4187-a4f6-5617b33fd734 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:30.790096 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb38d47-8e45-4187-a4f6-5617b33fd734 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:30.790479 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb38d47-8e45-4187-a4f6-5617b33fd734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:30.808112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1cb38d47-8e45-4187-a4f6-5617b33fd734 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:30.977176 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 852/1674] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:30.982146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d26500-5bac-4890-90ff-d2ddad9e8b6c None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:30.982437 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d26500-5bac-4890-90ff-d2ddad9e8b6c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:30.982717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d26500-5bac-4890-90ff-d2ddad9e8b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:31.003718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb38d47-8e45-4187-a4f6-5617b33fd734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:31.007213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb38d47-8e45-4187-a4f6-5617b33fd734 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:31.029725 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:31.046906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 823/1675] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:31.051591 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-05d26500-5bac-4890-90ff-d2ddad9e8b6c None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:31.055718 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-05d26500-5bac-4890-90ff-d2ddad9e8b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:31.086809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 853/1676] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:31.091577 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb38d47-8e45-4187-a4f6-5617b33fd734 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:31.107454 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb38d47-8e45-4187-a4f6-5617b33fd734 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:31.120578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2992d-941f-410b-bb48-9c8402a9528c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:31.121212 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2992d-941f-410b-bb48-9c8402a9528c admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:31.121493 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2992d-941f-410b-bb48-9c8402a9528c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:31.121769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2992d-941f-410b-bb48-9c8402a9528c admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:31.122428 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2992d-941f-410b-bb48-9c8402a9528c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:23:31.122736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2992d-941f-410b-bb48-9c8402a9528c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:31.126297 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2992d-941f-410b-bb48-9c8402a9528c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:31.159416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d26500-5bac-4890-90ff-d2ddad9e8b6c None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:31.162198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 824/1677] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:23:31.168672 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d26500-5bac-4890-90ff-d2ddad9e8b6c None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:31.181308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83dbfea-4cf3-4d66-b044-32fd4034425a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:31.182145 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83dbfea-4cf3-4d66-b044-32fd4034425a admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:31.182254 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83dbfea-4cf3-4d66-b044-32fd4034425a admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:31.182511 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83dbfea-4cf3-4d66-b044-32fd4034425a admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:31.183216 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83dbfea-4cf3-4d66-b044-32fd4034425a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:31.185174 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83dbfea-4cf3-4d66-b044-32fd4034425a admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:31.229001 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:31.273796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 854/1678] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:31 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:31.275340 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2992d-941f-410b-bb48-9c8402a9528c admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:31.290200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2992d-941f-410b-bb48-9c8402a9528c admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:31.466466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88b490-be39-4647-8408-acf7cff27c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:31.467799 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88b490-be39-4647-8408-acf7cff27c22 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:31.468234 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88b490-be39-4647-8408-acf7cff27c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:31.468648 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88b490-be39-4647-8408-acf7cff27c22 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:31.469446 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88b490-be39-4647-8408-acf7cff27c22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:31.471506 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88b490-be39-4647-8408-acf7cff27c22 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:31.519598 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 825/1679] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:31 2022] POST /identity/v3/users => generated 289 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:31.520958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83dbfea-4cf3-4d66-b044-32fd4034425a admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:31.542275 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83dbfea-4cf3-4d66-b044-32fd4034425a admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:31.556391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20919710-56f2-4d44-ae40-7bcaa4ab886d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:31.558152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20919710-56f2-4d44-ae40-7bcaa4ab886d admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:31.558511 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20919710-56f2-4d44-ae40-7bcaa4ab886d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:31.559344 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20919710-56f2-4d44-ae40-7bcaa4ab886d admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:31.560552 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20919710-56f2-4d44-ae40-7bcaa4ab886d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:31.565120 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20919710-56f2-4d44-ae40-7bcaa4ab886d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:31.589643 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 855/1680] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:31.591678 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88b490-be39-4647-8408-acf7cff27c22 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:31.605527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88b490-be39-4647-8408-acf7cff27c22 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:31.621349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231ede4-7c85-4541-b7c8-93c82556cfff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:31.621987 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231ede4-7c85-4541-b7c8-93c82556cfff admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:31.622278 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231ede4-7c85-4541-b7c8-93c82556cfff admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:31.622540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231ede4-7c85-4541-b7c8-93c82556cfff admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:31.623195 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231ede4-7c85-4541-b7c8-93c82556cfff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:31.624850 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231ede4-7c85-4541-b7c8-93c82556cfff admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:31.653635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 826/1681] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:31.657175 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20919710-56f2-4d44-ae40-7bcaa4ab886d admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:31.679201 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20919710-56f2-4d44-ae40-7bcaa4ab886d admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:31.695072 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea90b00-9676-42d5-a3da-4e49f998baa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:31.699116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea90b00-9676-42d5-a3da-4e49f998baa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:31.699664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea90b00-9676-42d5-a3da-4e49f998baa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:31.701148 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea90b00-9676-42d5-a3da-4e49f998baa6 admin admin] PATH_INFO: `/v3/projects/9b6e07889e3e42a4877fa56f8712ca94/users/9123f95e5acc448a9c4b4514f4a31bdb/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:31.731400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea90b00-9676-42d5-a3da-4e49f998baa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b6e07889e3e42a4877fa56f8712ca94, user_id=9123f95e5acc448a9c4b4514f4a31bdb, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:31.737828 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea90b00-9676-42d5-a3da-4e49f998baa6 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:31.765869 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 856/1682] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:31 2022] PUT /identity/v3/projects/9b6e07889e3e42a4877fa56f8712ca94/users/9123f95e5acc448a9c4b4514f4a31bdb/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:31.777030 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680f5b1-6325-4666-9ae8-2660f9e05aeb None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:31.777735 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680f5b1-6325-4666-9ae8-2660f9e05aeb None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:31.778143 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680f5b1-6325-4666-9ae8-2660f9e05aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:31.838343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9680f5b1-6325-4666-9ae8-2660f9e05aeb None None] MFA Rules not processed for user `9123f95e5acc448a9c4b4514f4a31bdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:31.840316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9680f5b1-6325-4666-9ae8-2660f9e05aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:32.007941 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 827/1683] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:32.012888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea90b00-9676-42d5-a3da-4e49f998baa6 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:32.134285 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea90b00-9676-42d5-a3da-4e49f998baa6 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:32.155606 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178d8f0-f6a4-488a-b450-c0691656e5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:32.156523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178d8f0-f6a4-488a-b450-c0691656e5fe admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:32.156881 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178d8f0-f6a4-488a-b450-c0691656e5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:32.157257 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178d8f0-f6a4-488a-b450-c0691656e5fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:32.158017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e178d8f0-f6a4-488a-b450-c0691656e5fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:32.163896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e178d8f0-f6a4-488a-b450-c0691656e5fe admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:32.246689 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 857/1684] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:32 2022] POST /identity/v3/projects => generated 379 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:32.252908 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680f5b1-6325-4666-9ae8-2660f9e05aeb None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:32.279606 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680f5b1-6325-4666-9ae8-2660f9e05aeb None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:32.455388 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:32.529052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d89e8-3ea8-46da-a163-c991d8de0be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:32.529692 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d89e8-3ea8-46da-a163-c991d8de0be9 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:32.529984 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d89e8-3ea8-46da-a163-c991d8de0be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:32.530247 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d89e8-3ea8-46da-a163-c991d8de0be9 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:32.532097 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d89e8-3ea8-46da-a163-c991d8de0be9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:32.534415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d89e8-3ea8-46da-a163-c991d8de0be9 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:32.596923 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 828/1685] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:32 2022] POST /identity/v3/users => generated 296 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:32.601898 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178d8f0-f6a4-488a-b450-c0691656e5fe admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:32.618725 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178d8f0-f6a4-488a-b450-c0691656e5fe admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:32.629801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75034ac7-562a-48d9-9d7c-e8495176bec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:32.630583 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75034ac7-562a-48d9-9d7c-e8495176bec6 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:32.630710 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75034ac7-562a-48d9-9d7c-e8495176bec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:32.631050 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75034ac7-562a-48d9-9d7c-e8495176bec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:32.631719 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75034ac7-562a-48d9-9d7c-e8495176bec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:32.633372 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75034ac7-562a-48d9-9d7c-e8495176bec6 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:32.653740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 858/1686] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:32.658268 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d89e8-3ea8-46da-a163-c991d8de0be9 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:32.675206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d89e8-3ea8-46da-a163-c991d8de0be9 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:32.689262 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7848928-fb22-4864-a25e-4c5c23af2073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:32.689969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7848928-fb22-4864-a25e-4c5c23af2073 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:32.690220 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7848928-fb22-4864-a25e-4c5c23af2073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:32.690490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7848928-fb22-4864-a25e-4c5c23af2073 admin admin] PATH_INFO: `/v3/projects/6a02a26bfc664cb4a9d0cbbd754647c7/users/f8a69f803cae4b7bb6ccce2943726ff4/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:32.721827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7848928-fb22-4864-a25e-4c5c23af2073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a02a26bfc664cb4a9d0cbbd754647c7, user_id=f8a69f803cae4b7bb6ccce2943726ff4, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:32.724587 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7848928-fb22-4864-a25e-4c5c23af2073 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:32.755168 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 829/1687] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:32 2022] PUT /identity/v3/projects/6a02a26bfc664cb4a9d0cbbd754647c7/users/f8a69f803cae4b7bb6ccce2943726ff4/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:32.762614 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75034ac7-562a-48d9-9d7c-e8495176bec6 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:32.779999 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75034ac7-562a-48d9-9d7c-e8495176bec6 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:32.925438 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3a786-d7e9-4641-ae50-6cd1bd9e64eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:32.928372 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3a786-d7e9-4641-ae50-6cd1bd9e64eb admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:32.928838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3a786-d7e9-4641-ae50-6cd1bd9e64eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:32.929266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3a786-d7e9-4641-ae50-6cd1bd9e64eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:32.930502 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3a786-d7e9-4641-ae50-6cd1bd9e64eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:32.933438 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3a786-d7e9-4641-ae50-6cd1bd9e64eb admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:32.950343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7848928-fb22-4864-a25e-4c5c23af2073 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:32.952345 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b7848928-fb22-4864-a25e-4c5c23af2073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:32.959954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 859/1688] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:32.961383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3a786-d7e9-4641-ae50-6cd1bd9e64eb admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:32.975489 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7848928-fb22-4864-a25e-4c5c23af2073 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:32.988059 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270795a8-b259-4e13-95cc-1f89752c38bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:32.988714 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270795a8-b259-4e13-95cc-1f89752c38bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:32.988991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270795a8-b259-4e13-95cc-1f89752c38bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:32.989103 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3a786-d7e9-4641-ae50-6cd1bd9e64eb admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:32.989260 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270795a8-b259-4e13-95cc-1f89752c38bb admin admin] PATH_INFO: `/v3/projects/62cdba2a8ecf40c7b320b495b3bf6c7d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:32.990438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270795a8-b259-4e13-95cc-1f89752c38bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=62cdba2a8ecf40c7b320b495b3bf6c7d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:32.992395 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270795a8-b259-4e13-95cc-1f89752c38bb admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:33.010934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b0035-81cc-4196-a646-d29e3ae0fe1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:33.013660 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b0035-81cc-4196-a646-d29e3ae0fe1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:33.013960 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b0035-81cc-4196-a646-d29e3ae0fe1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:33.015644 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b0035-81cc-4196-a646-d29e3ae0fe1f admin admin] PATH_INFO: `/v3/projects/6a02a26bfc664cb4a9d0cbbd754647c7/users/f8a69f803cae4b7bb6ccce2943726ff4/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:33.036012 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b0035-81cc-4196-a646-d29e3ae0fe1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a02a26bfc664cb4a9d0cbbd754647c7, user_id=f8a69f803cae4b7bb6ccce2943726ff4, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:33.038794 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b0035-81cc-4196-a646-d29e3ae0fe1f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:33.054647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 860/1689] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:32 2022] PUT /identity/v3/projects/6a02a26bfc664cb4a9d0cbbd754647c7/users/f8a69f803cae4b7bb6ccce2943726ff4/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:33.066579 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa552143-f755-4bdf-94ae-bbd962eefdec None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:33.067095 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa552143-f755-4bdf-94ae-bbd962eefdec None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:33.067518 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa552143-f755-4bdf-94ae-bbd962eefdec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:33.108199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-270795a8-b259-4e13-95cc-1f89752c38bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62cdba2a8ecf40c7b320b495b3bf6c7d', 'request_id': 'req-270795a8-b259-4e13-95cc-1f89752c38bb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:33.132284 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:33.134957 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-270795a8-b259-4e13-95cc-1f89752c38bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62cdba2a8ecf40c7b320b495b3bf6c7d', 'request_id': 'req-270795a8-b259-4e13-95cc-1f89752c38bb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:33.147451 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-aa552143-f755-4bdf-94ae-bbd962eefdec None None] MFA Rules not processed for user `f8a69f803cae4b7bb6ccce2943726ff4`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:33.152853 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-aa552143-f755-4bdf-94ae-bbd962eefdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:33.157994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-270795a8-b259-4e13-95cc-1f89752c38bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62cdba2a8ecf40c7b320b495b3bf6c7d', 'request_id': 'req-270795a8-b259-4e13-95cc-1f89752c38bb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:33.193527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-270795a8-b259-4e13-95cc-1f89752c38bb admin admin] The token cache is being invalidate because project 62cdba2a8ecf40c7b320b495b3bf6c7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:23:33.193781 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-270795a8-b259-4e13-95cc-1f89752c38bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:33.195115 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-270795a8-b259-4e13-95cc-1f89752c38bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:33.198646 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 830/1690] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:23:32 2022] DELETE /identity/v3/projects/62cdba2a8ecf40c7b320b495b3bf6c7d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:23:33.209302 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270795a8-b259-4e13-95cc-1f89752c38bb admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:33.221981 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-270795a8-b259-4e13-95cc-1f89752c38bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:33.303943 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 861/1691] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:33 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:33.304867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa552143-f755-4bdf-94ae-bbd962eefdec None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:33.306609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-aa552143-f755-4bdf-94ae-bbd962eefdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:33.455357 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270795a8-b259-4e13-95cc-1f89752c38bb admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:33.469360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc22c95-03eb-4c2e-82d5-afb60e220ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:33.470138 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc22c95-03eb-4c2e-82d5-afb60e220ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:33.470622 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc22c95-03eb-4c2e-82d5-afb60e220ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:33.470996 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc22c95-03eb-4c2e-82d5-afb60e220ad0 admin admin] PATH_INFO: `/v3/users/4dc1e2ef45fb41379c3ca66f5f83bdd9` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:33.472293 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc22c95-03eb-4c2e-82d5-afb60e220ad0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dc1e2ef45fb41379c3ca66f5f83bdd9)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:33.474359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc22c95-03eb-4c2e-82d5-afb60e220ad0 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:33.550525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa552143-f755-4bdf-94ae-bbd962eefdec None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:33.573765 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ba653-f8e6-41dc-8438-9b0f9fa49672 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:33.575150 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ba653-f8e6-41dc-8438-9b0f9fa49672 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:33.577625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ba653-f8e6-41dc-8438-9b0f9fa49672 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:33.577910 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ba653-f8e6-41dc-8438-9b0f9fa49672 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:33.578019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=82,keystone:INSERT=5,keystone:DELETE=13 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:33.584804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bcc22c95-03eb-4c2e-82d5-afb60e220ad0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dc1e2ef45fb41379c3ca66f5f83bdd9'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:33.606855 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bcc22c95-03eb-4c2e-82d5-afb60e220ad0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dc1e2ef45fb41379c3ca66f5f83bdd9'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:33.608537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bcc22c95-03eb-4c2e-82d5-afb60e220ad0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dc1e2ef45fb41379c3ca66f5f83bdd9'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:33.609564 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bcc22c95-03eb-4c2e-82d5-afb60e220ad0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dc1e2ef45fb41379c3ca66f5f83bdd9'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:33.615974 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ba653-f8e6-41dc-8438-9b0f9fa49672 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:33.618670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ba653-f8e6-41dc-8438-9b0f9fa49672 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:33.646275 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bcc22c95-03eb-4c2e-82d5-afb60e220ad0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dc1e2ef45fb41379c3ca66f5f83bdd9'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:33.671029 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 831/1692] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:23:33 2022] DELETE /identity/v3/users/4dc1e2ef45fb41379c3ca66f5f83bdd9 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:33.712730 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 862/1693] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:33 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:33.861169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc22c95-03eb-4c2e-82d5-afb60e220ad0 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:33.862857 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bcc22c95-03eb-4c2e-82d5-afb60e220ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:33.983283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc22c95-03eb-4c2e-82d5-afb60e220ad0 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:33.990567 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181dc6d-1694-4508-bb8c-0c98028cff67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:33.991199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181dc6d-1694-4508-bb8c-0c98028cff67 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:33.991482 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181dc6d-1694-4508-bb8c-0c98028cff67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:33.992114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181dc6d-1694-4508-bb8c-0c98028cff67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:33.997243 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4181dc6d-1694-4508-bb8c-0c98028cff67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:34.137291 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181dc6d-1694-4508-bb8c-0c98028cff67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:34.138763 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181dc6d-1694-4508-bb8c-0c98028cff67 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:34.160027 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 832/1694] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:34.390599 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d0831-99b4-4099-8f46-f3fc32eceff7 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:34.391141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d0831-99b4-4099-8f46-f3fc32eceff7 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:34.391570 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d0831-99b4-4099-8f46-f3fc32eceff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:34.466211 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-aa0d0831-99b4-4099-8f46-f3fc32eceff7 None None] MFA Rules not processed for user `aada82b4250745ab956466f29d1ddfb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:34.468635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-aa0d0831-99b4-4099-8f46-f3fc32eceff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:34.620094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 863/1695] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:34.631883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd0d69-43c7-472b-afbe-fe8191fd7332 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:34.632356 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd0d69-43c7-472b-afbe-fe8191fd7332 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:34.632433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd0d69-43c7-472b-afbe-fe8191fd7332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:34.668396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-15cd0d69-43c7-472b-afbe-fe8191fd7332 None None] MFA Rules not processed for user `aada82b4250745ab956466f29d1ddfb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:34.669665 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-15cd0d69-43c7-472b-afbe-fe8191fd7332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:34.695502 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 833/1696] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:34.702454 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d0831-99b4-4099-8f46-f3fc32eceff7 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:34.704206 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-aa0d0831-99b4-4099-8f46-f3fc32eceff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:34.811380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:34.953503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeba928-b933-4f5c-aa66-0b2bf9131160 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:34.953616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d0831-99b4-4099-8f46-f3fc32eceff7 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:34.953710 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeba928-b933-4f5c-aa66-0b2bf9131160 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:34.953852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeba928-b933-4f5c-aa66-0b2bf9131160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:34.968708 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8160221-2ec7-4919-b650-5cd90351597d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:34.969323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8160221-2ec7-4919-b650-5cd90351597d service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:34.969697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8160221-2ec7-4919-b650-5cd90351597d service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:34.970072 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8160221-2ec7-4919-b650-5cd90351597d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:35.005011 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8160221-2ec7-4919-b650-5cd90351597d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:35.007324 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8160221-2ec7-4919-b650-5cd90351597d service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:35.040512 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6eeba928-b933-4f5c-aa66-0b2bf9131160 None None] MFA Rules not processed for user `07cc7bb8955e4255a2a9d9d717f06413`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:35.042576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6eeba928-b933-4f5c-aa66-0b2bf9131160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:35.051821 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 864/1697] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:35.178132 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:35.187435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 834/1698] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:34 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:23:35.187869 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8160221-2ec7-4919-b650-5cd90351597d service nova] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:35.189874 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e8160221-2ec7-4919-b650-5cd90351597d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:35.334400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8160221-2ec7-4919-b650-5cd90351597d service nova] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:35.348190 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120935c5-498c-42b5-bed8-8bdd74e0dbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:35.348817 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120935c5-498c-42b5-bed8-8bdd74e0dbba admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:35.349103 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120935c5-498c-42b5-bed8-8bdd74e0dbba admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:35.349381 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120935c5-498c-42b5-bed8-8bdd74e0dbba admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:35.350028 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120935c5-498c-42b5-bed8-8bdd74e0dbba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:35.351958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120935c5-498c-42b5-bed8-8bdd74e0dbba admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:35.432835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeba928-b933-4f5c-aa66-0b2bf9131160 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:35.434921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 865/1699] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:35 2022] POST /identity/v3/projects => generated 405 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:35.447596 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeba928-b933-4f5c-aa66-0b2bf9131160 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:35.658576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39b58b-73ca-4502-930a-7e18e49f7d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:35.660093 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39b58b-73ca-4502-930a-7e18e49f7d73 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:35.660690 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39b58b-73ca-4502-930a-7e18e49f7d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:35.661152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39b58b-73ca-4502-930a-7e18e49f7d73 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:35.662429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39b58b-73ca-4502-930a-7e18e49f7d73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:35.665045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39b58b-73ca-4502-930a-7e18e49f7d73 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:35.710653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 835/1700] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:35 2022] POST /identity/v3/users => generated 309 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:35.714900 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120935c5-498c-42b5-bed8-8bdd74e0dbba admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:35.727858 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120935c5-498c-42b5-bed8-8bdd74e0dbba admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:35.742600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7ccef-b442-4842-ac9e-a5f9de05828a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:35.743877 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7ccef-b442-4842-ac9e-a5f9de05828a admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:35.745626 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7ccef-b442-4842-ac9e-a5f9de05828a admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:35.746102 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7ccef-b442-4842-ac9e-a5f9de05828a admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:35.747566 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7ccef-b442-4842-ac9e-a5f9de05828a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:35.751531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7ccef-b442-4842-ac9e-a5f9de05828a admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:35.774764 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 866/1701] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:35.778969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39b58b-73ca-4502-930a-7e18e49f7d73 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:35.809946 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39b58b-73ca-4502-930a-7e18e49f7d73 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:35.817875 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0762d0-4a3a-406f-be75-ce7f26ba3dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:35.819536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0762d0-4a3a-406f-be75-ce7f26ba3dd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:35.820048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0762d0-4a3a-406f-be75-ce7f26ba3dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:35.820553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0762d0-4a3a-406f-be75-ce7f26ba3dd8 admin admin] PATH_INFO: `/v3/projects/7f876df472184339907077a54d688fc3/users/a8733f0e3a6242799d1f0bb6a4da01cb/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:35.843052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0762d0-4a3a-406f-be75-ce7f26ba3dd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f876df472184339907077a54d688fc3, user_id=a8733f0e3a6242799d1f0bb6a4da01cb, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:35.848057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0762d0-4a3a-406f-be75-ce7f26ba3dd8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:35.870153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 836/1702] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:35 2022] PUT /identity/v3/projects/7f876df472184339907077a54d688fc3/users/a8733f0e3a6242799d1f0bb6a4da01cb/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:35.875156 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7ccef-b442-4842-ac9e-a5f9de05828a admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:35.897901 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7ccef-b442-4842-ac9e-a5f9de05828a admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:36.048533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7f905-febc-49ce-a57f-bb275dc98d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:36.049307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7f905-febc-49ce-a57f-bb275dc98d46 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:36.049688 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7f905-febc-49ce-a57f-bb275dc98d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:36.050068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7f905-febc-49ce-a57f-bb275dc98d46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:36.050906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7f905-febc-49ce-a57f-bb275dc98d46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:36.053916 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7f905-febc-49ce-a57f-bb275dc98d46 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:36.098672 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 867/1703] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:36.099195 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0762d0-4a3a-406f-be75-ce7f26ba3dd8 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:36.118505 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0762d0-4a3a-406f-be75-ce7f26ba3dd8 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:36.133794 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b7067-1ae6-495f-8997-6cf632919749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:36.136533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b7067-1ae6-495f-8997-6cf632919749 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:36.136764 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b7067-1ae6-495f-8997-6cf632919749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:36.136979 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b7067-1ae6-495f-8997-6cf632919749 admin admin] PATH_INFO: `/v3/projects/7f876df472184339907077a54d688fc3/users/a8733f0e3a6242799d1f0bb6a4da01cb/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:36.141904 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b7067-1ae6-495f-8997-6cf632919749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f876df472184339907077a54d688fc3, user_id=a8733f0e3a6242799d1f0bb6a4da01cb, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:36.152011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b7067-1ae6-495f-8997-6cf632919749 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:36.205318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 837/1704] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:36 2022] PUT /identity/v3/projects/7f876df472184339907077a54d688fc3/users/a8733f0e3a6242799d1f0bb6a4da01cb/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:23:37.033637 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b7ebf-fd55-4665-9a0d-cd54dd77cf15 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:37.034347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b7ebf-fd55-4665-9a0d-cd54dd77cf15 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:37.034732 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b7ebf-fd55-4665-9a0d-cd54dd77cf15 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:37.036702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 868/1705] 10.222.0.81 () {54 vars in 912 bytes} [Mon Sep 19 11:23:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:23:37.040200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b7067-1ae6-495f-8997-6cf632919749 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:37.043172 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0b5b7067-1ae6-495f-8997-6cf632919749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:37.043731 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf6cb7-5cae-4ecc-be27-a5d8aa6cf38b None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:37.044148 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf6cb7-5cae-4ecc-be27-a5d8aa6cf38b None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:37.044939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf6cb7-5cae-4ecc-be27-a5d8aa6cf38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:37.159084 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-30cf6cb7-5cae-4ecc-be27-a5d8aa6cf38b None None] MFA Rules not processed for user `65de2e5f67dd4bb6a20cf3934afc8abd`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:37.163744 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-30cf6cb7-5cae-4ecc-be27-a5d8aa6cf38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:37.209967 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:37.235900 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b7067-1ae6-495f-8997-6cf632919749 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:37.254663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9464b39-a107-4cc5-b38d-07e65c7ee066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:37.255709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9464b39-a107-4cc5-b38d-07e65c7ee066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:37.256301 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9464b39-a107-4cc5-b38d-07e65c7ee066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:37.256727 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9464b39-a107-4cc5-b38d-07e65c7ee066 admin admin] PATH_INFO: `/v3/projects/193bbbe154d74a668ef975a9f2f4bace` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:37.272028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9464b39-a107-4cc5-b38d-07e65c7ee066 admin admin] RBAC: Authorizing `identity:delete_project(project_id=193bbbe154d74a668ef975a9f2f4bace)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:37.277652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9464b39-a107-4cc5-b38d-07e65c7ee066 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:37.380862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b9464b39-a107-4cc5-b38d-07e65c7ee066 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '193bbbe154d74a668ef975a9f2f4bace', 'request_id': 'req-b9464b39-a107-4cc5-b38d-07e65c7ee066'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:37.387115 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:37.411445 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b9464b39-a107-4cc5-b38d-07e65c7ee066 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '193bbbe154d74a668ef975a9f2f4bace', 'request_id': 'req-b9464b39-a107-4cc5-b38d-07e65c7ee066'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:37.450129 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b9464b39-a107-4cc5-b38d-07e65c7ee066 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '193bbbe154d74a668ef975a9f2f4bace', 'request_id': 'req-b9464b39-a107-4cc5-b38d-07e65c7ee066'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:37.471799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 869/1706] 10.222.0.81 () {58 vars in 1009 bytes} [Mon Sep 19 11:23:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:37.479149 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b9464b39-a107-4cc5-b38d-07e65c7ee066 admin admin] The token cache is being invalidate because project 193bbbe154d74a668ef975a9f2f4bace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:23:37.482187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b9464b39-a107-4cc5-b38d-07e65c7ee066 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:37.483319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b9464b39-a107-4cc5-b38d-07e65c7ee066 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:37.488556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 838/1707] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:23:37 2022] DELETE /identity/v3/projects/193bbbe154d74a668ef975a9f2f4bace => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:37.495285 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1313d22-a70e-4c03-b989-29c2c056db48 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:37.495578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1313d22-a70e-4c03-b989-29c2c056db48 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:37.495916 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1313d22-a70e-4c03-b989-29c2c056db48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:37.538660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9464b39-a107-4cc5-b38d-07e65c7ee066 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:37.540888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b9464b39-a107-4cc5-b38d-07e65c7ee066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:37.553796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-d1313d22-a70e-4c03-b989-29c2c056db48 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:37.555815 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-d1313d22-a70e-4c03-b989-29c2c056db48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:37.753735 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 870/1708] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:37.765018 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da503e21-010c-4180-a455-f09ba50cb4dd None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:37.765350 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da503e21-010c-4180-a455-f09ba50cb4dd None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:37.765897 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da503e21-010c-4180-a455-f09ba50cb4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:37.771918 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9464b39-a107-4cc5-b38d-07e65c7ee066 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:37.791235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd9ce2-dde6-4c8b-be64-3e6d5fe8924d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:37.793336 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd9ce2-dde6-4c8b-be64-3e6d5fe8924d service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:37.794693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd9ce2-dde6-4c8b-be64-3e6d5fe8924d service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:37.795408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd9ce2-dde6-4c8b-be64-3e6d5fe8924d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:37.800888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-43bd9ce2-dde6-4c8b-be64-3e6d5fe8924d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:37.850145 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd9ce2-dde6-4c8b-be64-3e6d5fe8924d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:37.855393 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-da503e21-010c-4180-a455-f09ba50cb4dd None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:37.858318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd9ce2-dde6-4c8b-be64-3e6d5fe8924d service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:37.859518 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-da503e21-010c-4180-a455-f09ba50cb4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:37.892776 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 839/1709] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:37.899885 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 871/1710] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:37.903364 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd9ce2-dde6-4c8b-be64-3e6d5fe8924d service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:37.931188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd9ce2-dde6-4c8b-be64-3e6d5fe8924d service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:37.957483 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578524a4-e809-463c-929d-a997f90e5da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:37.958341 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578524a4-e809-463c-929d-a997f90e5da6 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:37.960757 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578524a4-e809-463c-929d-a997f90e5da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:37.961153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578524a4-e809-463c-929d-a997f90e5da6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:37.961948 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578524a4-e809-463c-929d-a997f90e5da6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:23:37.962358 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578524a4-e809-463c-929d-a997f90e5da6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:37.966506 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578524a4-e809-463c-929d-a997f90e5da6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:38.019072 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 840/1711] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:23:38.022432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da503e21-010c-4180-a455-f09ba50cb4dd None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:38.039696 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da503e21-010c-4180-a455-f09ba50cb4dd None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:38.058086 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ceff57-485d-49fe-b9ac-b6e5b8973ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:38.058726 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ceff57-485d-49fe-b9ac-b6e5b8973ccc admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:38.059011 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ceff57-485d-49fe-b9ac-b6e5b8973ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:38.059283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ceff57-485d-49fe-b9ac-b6e5b8973ccc admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:38.065357 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ceff57-485d-49fe-b9ac-b6e5b8973ccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:38.067815 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ceff57-485d-49fe-b9ac-b6e5b8973ccc admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:38.141806 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 872/1712] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:38 2022] POST /identity/v3/projects => generated 369 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:38.145864 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578524a4-e809-463c-929d-a997f90e5da6 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:38.161397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578524a4-e809-463c-929d-a997f90e5da6 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:38.360297 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4d3f1-710c-4afa-9e10-a580fbf81fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:38.363351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4d3f1-710c-4afa-9e10-a580fbf81fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:38.364241 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4d3f1-710c-4afa-9e10-a580fbf81fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:38.364742 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4d3f1-710c-4afa-9e10-a580fbf81fa0 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:38.365823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4d3f1-710c-4afa-9e10-a580fbf81fa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:38.370284 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4d3f1-710c-4afa-9e10-a580fbf81fa0 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:38.435618 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ceff57-485d-49fe-b9ac-b6e5b8973ccc admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:38.436146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 841/1713] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:38 2022] POST /identity/v3/users => generated 285 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:38.462195 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ceff57-485d-49fe-b9ac-b6e5b8973ccc admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:38.486302 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82248a8f-9fda-4900-92b1-c5bce02ac0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:38.487212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82248a8f-9fda-4900-92b1-c5bce02ac0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:38.487649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82248a8f-9fda-4900-92b1-c5bce02ac0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:38.488103 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82248a8f-9fda-4900-92b1-c5bce02ac0f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:38.488905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82248a8f-9fda-4900-92b1-c5bce02ac0f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:38.496717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82248a8f-9fda-4900-92b1-c5bce02ac0f9 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:38.523559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 873/1714] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:38.527301 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4d3f1-710c-4afa-9e10-a580fbf81fa0 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:38.544299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4d3f1-710c-4afa-9e10-a580fbf81fa0 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:38.557517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea5047e-2dbb-4f84-affb-4f3394bebe40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:38.558351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5047e-2dbb-4f84-affb-4f3394bebe40 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:38.558734 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5047e-2dbb-4f84-affb-4f3394bebe40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:38.559109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5047e-2dbb-4f84-affb-4f3394bebe40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:38.559984 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea5047e-2dbb-4f84-affb-4f3394bebe40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:38.563668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea5047e-2dbb-4f84-affb-4f3394bebe40 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:38.602066 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 842/1715] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:38.605653 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82248a8f-9fda-4900-92b1-c5bce02ac0f9 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:38.630154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82248a8f-9fda-4900-92b1-c5bce02ac0f9 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:38.637469 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c4b8b-6379-4e3e-a8d4-3b7a0df18867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:38.638719 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c4b8b-6379-4e3e-a8d4-3b7a0df18867 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:38.639309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c4b8b-6379-4e3e-a8d4-3b7a0df18867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:38.639981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c4b8b-6379-4e3e-a8d4-3b7a0df18867 admin admin] PATH_INFO: `/v3/projects/34f46c1ef9e5469a8585aabd8cd56bf5/users/e091cd41dab940fdb56ea9179d2298bb/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:38.683592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c4b8b-6379-4e3e-a8d4-3b7a0df18867 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f46c1ef9e5469a8585aabd8cd56bf5, user_id=e091cd41dab940fdb56ea9179d2298bb, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:38.687220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c4b8b-6379-4e3e-a8d4-3b7a0df18867 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:38.728349 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 874/1716] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:38 2022] PUT /identity/v3/projects/34f46c1ef9e5469a8585aabd8cd56bf5/users/e091cd41dab940fdb56ea9179d2298bb/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:38.730785 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed03be-a599-420b-9e6d-fc624d09c6b1 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:38.731069 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed03be-a599-420b-9e6d-fc624d09c6b1 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:38.731342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed03be-a599-420b-9e6d-fc624d09c6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:38.841098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba1ec6-80b9-44e8-a439-dd8979eb6afa None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:38.841452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba1ec6-80b9-44e8-a439-dd8979eb6afa None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:38.841997 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba1ec6-80b9-44e8-a439-dd8979eb6afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:38.858772 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f5ed03be-a599-420b-9e6d-fc624d09c6b1 None None] MFA Rules not processed for user `e091cd41dab940fdb56ea9179d2298bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:38.863615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f5ed03be-a599-420b-9e6d-fc624d09c6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:38.911191 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-bdba1ec6-80b9-44e8-a439-dd8979eb6afa None None] MFA Rules not processed for user `9a4a7aca62974db38138a38de399139f`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:38.919515 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-bdba1ec6-80b9-44e8-a439-dd8979eb6afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:39.012070 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:39.046519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:39.099251 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 875/1717] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:39.103067 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ec933-e3af-4184-95d2-058680366629 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:39.103359 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ec933-e3af-4184-95d2-058680366629 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:39.104234 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ec933-e3af-4184-95d2-058680366629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:39.120318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 843/1718] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:23:39.129635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed03be-a599-420b-9e6d-fc624d09c6b1 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:39.139166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f5ed03be-a599-420b-9e6d-fc624d09c6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:39.208070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-de8ec933-e3af-4184-95d2-058680366629 None None] MFA Rules not processed for user `9f160368e1984319ad4fa37c2794a956`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:39.209668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-de8ec933-e3af-4184-95d2-058680366629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:39.361548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed03be-a599-420b-9e6d-fc624d09c6b1 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:39.375038 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 876/1719] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:39.375924 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ec933-e3af-4184-95d2-058680366629 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:39.377035 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf1e64-136e-4662-9ea6-5d53e79d29bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:39.378053 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf1e64-136e-4662-9ea6-5d53e79d29bc service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:39.378330 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf1e64-136e-4662-9ea6-5d53e79d29bc service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:39.378715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf1e64-136e-4662-9ea6-5d53e79d29bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:39.396676 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf1e64-136e-4662-9ea6-5d53e79d29bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:39.408486 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf1e64-136e-4662-9ea6-5d53e79d29bc service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:39.442225 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 844/1720] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:39.457668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f08031-e1c6-4293-a9ee-558fd8fb889f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:39.458093 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f08031-e1c6-4293-a9ee-558fd8fb889f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:39.458497 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f08031-e1c6-4293-a9ee-558fd8fb889f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:39.553429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-44f08031-e1c6-4293-a9ee-558fd8fb889f None None] MFA Rules not processed for user `9f160368e1984319ad4fa37c2794a956`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:39.555242 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-44f08031-e1c6-4293-a9ee-558fd8fb889f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:39.557184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ec933-e3af-4184-95d2-058680366629 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:39.574663 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a21d3-c21c-486e-84b1-28414bb56d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:39.577965 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a21d3-c21c-486e-84b1-28414bb56d72 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:39.578235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a21d3-c21c-486e-84b1-28414bb56d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:39.578520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a21d3-c21c-486e-84b1-28414bb56d72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:39.579285 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a21d3-c21c-486e-84b1-28414bb56d72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:39.580542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 845/1721] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:39.582103 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e946543-b893-4cd4-bf00-d1bba71c5b84 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:39.585393 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e946543-b893-4cd4-bf00-d1bba71c5b84 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:39.585809 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e946543-b893-4cd4-bf00-d1bba71c5b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:39.597535 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a21d3-c21c-486e-84b1-28414bb56d72 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:39.657131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-2e946543-b893-4cd4-bf00-d1bba71c5b84 None None] MFA Rules not processed for user `a8733f0e3a6242799d1f0bb6a4da01cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:39.658572 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2e946543-b893-4cd4-bf00-d1bba71c5b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:39.727738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 877/1722] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:39 2022] POST /identity/v3/projects => generated 369 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:39.729746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a21d3-c21c-486e-84b1-28414bb56d72 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:39.817530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 846/1723] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:39 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:39.818419 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e946543-b893-4cd4-bf00-d1bba71c5b84 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:39.823057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2e946543-b893-4cd4-bf00-d1bba71c5b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:39.962099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:39.971927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a21d3-c21c-486e-84b1-28414bb56d72 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:39.988472 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892085b5-3e3c-4699-bba6-4b5c02f082d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:39.989518 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892085b5-3e3c-4699-bba6-4b5c02f082d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:39.990020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892085b5-3e3c-4699-bba6-4b5c02f082d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:39.990511 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892085b5-3e3c-4699-bba6-4b5c02f082d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:39.992995 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892085b5-3e3c-4699-bba6-4b5c02f082d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:40.004292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892085b5-3e3c-4699-bba6-4b5c02f082d0 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:40.047657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:40.058927 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e946543-b893-4cd4-bf00-d1bba71c5b84 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:40.067966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 878/1724] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:39 2022] POST /identity/v3/users => generated 291 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:40.083663 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892085b5-3e3c-4699-bba6-4b5c02f082d0 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:40.347118 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a486e-2102-4e51-945d-e06bec6b04bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e7a18dc4a4d4d35a05f1aafbaaf9c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:40.348347 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a486e-2102-4e51-945d-e06bec6b04bc service placement] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:40.348632 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a486e-2102-4e51-945d-e06bec6b04bc service placement] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:40.348903 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a486e-2102-4e51-945d-e06bec6b04bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:40.351972 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d91a486e-2102-4e51-945d-e06bec6b04bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:40.393545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a486e-2102-4e51-945d-e06bec6b04bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:40.396382 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a486e-2102-4e51-945d-e06bec6b04bc service placement] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:40.424088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 847/1725] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:40.424989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a486e-2102-4e51-945d-e06bec6b04bc service placement] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:40.453369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a486e-2102-4e51-945d-e06bec6b04bc service placement] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:40.458768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21254ad-efbf-40a3-8364-a308eeeeac67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:40.459645 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21254ad-efbf-40a3-8364-a308eeeeac67 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:40.460109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21254ad-efbf-40a3-8364-a308eeeeac67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:40.460653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21254ad-efbf-40a3-8364-a308eeeeac67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:40.461307 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21254ad-efbf-40a3-8364-a308eeeeac67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:40.465868 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21254ad-efbf-40a3-8364-a308eeeeac67 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:40.489731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 848/1726] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:40.491900 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21254ad-efbf-40a3-8364-a308eeeeac67 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:40.495367 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892085b5-3e3c-4699-bba6-4b5c02f082d0 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:40.527183 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21254ad-efbf-40a3-8364-a308eeeeac67 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:40.528324 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84106a-acac-453c-87a2-c97c81a278dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:40.529040 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84106a-acac-453c-87a2-c97c81a278dd service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:40.529504 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84106a-acac-453c-87a2-c97c81a278dd service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:40.529875 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84106a-acac-453c-87a2-c97c81a278dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:40.546071 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52621043-c89e-4c93-82c2-b3b0766ecf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:40.546903 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52621043-c89e-4c93-82c2-b3b0766ecf57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:40.547185 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52621043-c89e-4c93-82c2-b3b0766ecf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:40.547457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52621043-c89e-4c93-82c2-b3b0766ecf57 admin admin] PATH_INFO: `/v3/projects/3daf7ecec0a34c6585b9e0a950b5f5ca/users/c170ecbab1fc4916905ecc064dc69cae/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:40.577416 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52621043-c89e-4c93-82c2-b3b0766ecf57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3daf7ecec0a34c6585b9e0a950b5f5ca, user_id=c170ecbab1fc4916905ecc064dc69cae, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:40.586036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52621043-c89e-4c93-82c2-b3b0766ecf57 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:40.622859 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 849/1727] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:40 2022] PUT /identity/v3/projects/3daf7ecec0a34c6585b9e0a950b5f5ca/users/c170ecbab1fc4916905ecc064dc69cae/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:40.626751 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52621043-c89e-4c93-82c2-b3b0766ecf57 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:40.764129 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52621043-c89e-4c93-82c2-b3b0766ecf57 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:40.769148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ba231-150e-4c20-a862-4dc91f34adf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:40.770144 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ba231-150e-4c20-a862-4dc91f34adf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:40.770144 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ba231-150e-4c20-a862-4dc91f34adf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:40.770354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ba231-150e-4c20-a862-4dc91f34adf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:40.771595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ba231-150e-4c20-a862-4dc91f34adf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:40.773112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ba231-150e-4c20-a862-4dc91f34adf2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:40.792266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84106a-acac-453c-87a2-c97c81a278dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:40.795640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 850/1728] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:40.797808 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84106a-acac-453c-87a2-c97c81a278dd service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:40.805597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ba231-150e-4c20-a862-4dc91f34adf2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:40.810874 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 879/1729] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:40 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 742 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:40.834753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ba231-150e-4c20-a862-4dc91f34adf2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:40.845352 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e76e4f-bd55-4a52-b1d7-cf3aab32da87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:40.846082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e76e4f-bd55-4a52-b1d7-cf3aab32da87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:40.846364 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e76e4f-bd55-4a52-b1d7-cf3aab32da87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:40.846685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e76e4f-bd55-4a52-b1d7-cf3aab32da87 admin admin] PATH_INFO: `/v3/projects/3daf7ecec0a34c6585b9e0a950b5f5ca/users/c170ecbab1fc4916905ecc064dc69cae/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:40.848794 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e76e4f-bd55-4a52-b1d7-cf3aab32da87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3daf7ecec0a34c6585b9e0a950b5f5ca, user_id=c170ecbab1fc4916905ecc064dc69cae, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:40.855654 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e76e4f-bd55-4a52-b1d7-cf3aab32da87 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:40.893626 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 851/1730] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:40 2022] PUT /identity/v3/projects/3daf7ecec0a34c6585b9e0a950b5f5ca/users/c170ecbab1fc4916905ecc064dc69cae/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:40.895411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447c8fa-9ffb-4e24-b188-e2352c2a283e None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:40.895825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447c8fa-9ffb-4e24-b188-e2352c2a283e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:40.896209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447c8fa-9ffb-4e24-b188-e2352c2a283e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:40.982199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-2447c8fa-9ffb-4e24-b188-e2352c2a283e None None] MFA Rules not processed for user `c170ecbab1fc4916905ecc064dc69cae`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:40.984093 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2447c8fa-9ffb-4e24-b188-e2352c2a283e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:41.189410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 880/1731] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:41.204073 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e76e4f-bd55-4a52-b1d7-cf3aab32da87 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:41.206184 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a8e76e4f-bd55-4a52-b1d7-cf3aab32da87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:41.271826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:41.363532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e76e4f-bd55-4a52-b1d7-cf3aab32da87 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:41.383632 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40d557-2405-46db-98d3-214e7df8c98d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:41.383632 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40d557-2405-46db-98d3-214e7df8c98d service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:41.383632 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40d557-2405-46db-98d3-214e7df8c98d service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:41.383632 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40d557-2405-46db-98d3-214e7df8c98d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:41.398748 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40d557-2405-46db-98d3-214e7df8c98d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:41.403148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40d557-2405-46db-98d3-214e7df8c98d service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:41.427197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 852/1732] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:41.566787 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447c8fa-9ffb-4e24-b188-e2352c2a283e None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:41.568621 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2447c8fa-9ffb-4e24-b188-e2352c2a283e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:41.598425 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447c8fa-9ffb-4e24-b188-e2352c2a283e None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:41.611316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba3f6e-383d-41ee-a954-2116188bc719 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:41.615481 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba3f6e-383d-41ee-a954-2116188bc719 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:41.618890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba3f6e-383d-41ee-a954-2116188bc719 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:41.619416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba3f6e-383d-41ee-a954-2116188bc719 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:41.667429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:41.772290 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938cbf7-e82a-473e-8df5-1a0c8c2ce05d None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:41.772940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938cbf7-e82a-473e-8df5-1a0c8c2ce05d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:41.773419 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938cbf7-e82a-473e-8df5-1a0c8c2ce05d None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:41.775319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 853/1733] 10.222.0.81 () {54 vars in 912 bytes} [Mon Sep 19 11:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:23:41.781319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7d384-45de-479e-9d02-c49ca4865134 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:41.781908 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7d384-45de-479e-9d02-c49ca4865134 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:41.782644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7d384-45de-479e-9d02-c49ca4865134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:41.798891 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba3f6e-383d-41ee-a954-2116188bc719 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:41.803517 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba3f6e-383d-41ee-a954-2116188bc719 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:41.834626 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d8d7d384-45de-479e-9d02-c49ca4865134 None None] MFA Rules not processed for user `65de2e5f67dd4bb6a20cf3934afc8abd`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:41.836227 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d8d7d384-45de-479e-9d02-c49ca4865134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:41.842131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 881/1734] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:42.004640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 854/1735] 10.222.0.81 () {58 vars in 1009 bytes} [Mon Sep 19 11:23:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:42.065309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba3f6e-383d-41ee-a954-2116188bc719 service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:42.086494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba3f6e-383d-41ee-a954-2116188bc719 service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:42.107119 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb4b41-c312-4c8f-ba44-a605174b2690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:42.110654 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb4b41-c312-4c8f-ba44-a605174b2690 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:42.110654 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb4b41-c312-4c8f-ba44-a605174b2690 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:42.113862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb4b41-c312-4c8f-ba44-a605174b2690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:42.152294 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb4b41-c312-4c8f-ba44-a605174b2690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:42.155952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb4b41-c312-4c8f-ba44-a605174b2690 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:42.196879 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 882/1736] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:42.210023 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df2c1a-eadc-432b-9c6b-00760641b92d None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:42.210317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df2c1a-eadc-432b-9c6b-00760641b92d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:42.210591 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df2c1a-eadc-432b-9c6b-00760641b92d None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:42.213318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 855/1737] 10.222.0.81 () {54 vars in 921 bytes} [Mon Sep 19 11:23:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:23:42.242716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca6110-4428-4789-8c87-8167038b922e None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:42.243038 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca6110-4428-4789-8c87-8167038b922e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:42.243286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca6110-4428-4789-8c87-8167038b922e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:42.291277 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-f7ca6110-4428-4789-8c87-8167038b922e None None] MFA Rules not processed for user `6e7a18dc4a4d4d35a05f1aafbaaf9c31`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:42.298499 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f7ca6110-4428-4789-8c87-8167038b922e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:42.533149 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 883/1738] 10.222.0.81 () {58 vars in 1018 bytes} [Mon Sep 19 11:23:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:42.593748 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f304c-ef58-4182-a119-6081e9a35d5d None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:42.594277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f304c-ef58-4182-a119-6081e9a35d5d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:42.594450 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f304c-ef58-4182-a119-6081e9a35d5d None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:42.596151 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 856/1739] 10.222.0.81 () {54 vars in 961 bytes} [Mon Sep 19 11:23:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:23:42.611692 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502e072-9d4d-4d9e-b5d2-cb1725f81c54 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:42.611692 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502e072-9d4d-4d9e-b5d2-cb1725f81c54 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:42.611692 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502e072-9d4d-4d9e-b5d2-cb1725f81c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:42.700768 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-b502e072-9d4d-4d9e-b5d2-cb1725f81c54 None None] MFA Rules not processed for user `6e7a18dc4a4d4d35a05f1aafbaaf9c31`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:42.711337 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b502e072-9d4d-4d9e-b5d2-cb1725f81c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:42.771368 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 884/1740] 10.222.0.81 () {58 vars in 1058 bytes} [Mon Sep 19 11:23:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:42.781322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f304c-ef58-4182-a119-6081e9a35d5d None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:42.803472 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f304c-ef58-4182-a119-6081e9a35d5d None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:42.810852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9299fee-151e-498a-8350-3238ae9a340a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e7a18dc4a4d4d35a05f1aafbaaf9c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:42.811630 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9299fee-151e-498a-8350-3238ae9a340a service placement] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:42.812018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9299fee-151e-498a-8350-3238ae9a340a service placement] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:42.812613 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9299fee-151e-498a-8350-3238ae9a340a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:42.824133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9299fee-151e-498a-8350-3238ae9a340a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:42.825711 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9299fee-151e-498a-8350-3238ae9a340a service placement] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:42.849693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 857/1741] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:43.583023 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f258e98-4a85-49c5-aecc-bb4413062f0e None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:43.583625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f258e98-4a85-49c5-aecc-bb4413062f0e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:43.583625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f258e98-4a85-49c5-aecc-bb4413062f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:43.710147 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-0f258e98-4a85-49c5-aecc-bb4413062f0e None None] MFA Rules not processed for user `a3097d9c130746ca840bf85f33bc006b`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:43.711427 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-0f258e98-4a85-49c5-aecc-bb4413062f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:43.831839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 885/1742] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:43.848633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9299fee-151e-498a-8350-3238ae9a340a service placement] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:43.983829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcce37-6e23-45e8-872c-1954c7a02398 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:43.984275 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcce37-6e23-45e8-872c-1954c7a02398 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:43.984662 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcce37-6e23-45e8-872c-1954c7a02398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:44.038847 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-6dfcce37-6e23-45e8-872c-1954c7a02398 None None] MFA Rules not processed for user `9f160368e1984319ad4fa37c2794a956`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:44.040231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-6dfcce37-6e23-45e8-872c-1954c7a02398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:44.067163 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:44.100712 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9299fee-151e-498a-8350-3238ae9a340a service placement] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:44.115506 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ca3ca-d2a0-4c1a-bc1b-ebc65c6bd3a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:44.116172 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ca3ca-d2a0-4c1a-bc1b-ebc65c6bd3a1 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:44.116470 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ca3ca-d2a0-4c1a-bc1b-ebc65c6bd3a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:44.116734 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ca3ca-d2a0-4c1a-bc1b-ebc65c6bd3a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:44.143743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ca3ca-d2a0-4c1a-bc1b-ebc65c6bd3a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:44.146663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ca3ca-d2a0-4c1a-bc1b-ebc65c6bd3a1 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:44.170443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 886/1743] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:44.188812 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 858/1744] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:44.292242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235e3ee-c87e-4951-8b75-3a52b537fd21 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:44.292713 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235e3ee-c87e-4951-8b75-3a52b537fd21 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:44.293385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235e3ee-c87e-4951-8b75-3a52b537fd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:44.325089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ca3ca-d2a0-4c1a-bc1b-ebc65c6bd3a1 service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:44.326852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e73ca3ca-d2a0-4c1a-bc1b-ebc65c6bd3a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:44.455752 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-4235e3ee-c87e-4951-8b75-3a52b537fd21 None None] MFA Rules not processed for user `06d279b488d148428b7a3c35a9786ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:44.457017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-4235e3ee-c87e-4951-8b75-3a52b537fd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:44.535846 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:44.601162 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ca3ca-d2a0-4c1a-bc1b-ebc65c6bd3a1 service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:44.647250 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b372a6-9094-41fc-86b3-6052a2323255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:44.647938 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b372a6-9094-41fc-86b3-6052a2323255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:44.648222 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b372a6-9094-41fc-86b3-6052a2323255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:44.652092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b372a6-9094-41fc-86b3-6052a2323255 admin admin] PATH_INFO: `/v3/users/07cc7bb8955e4255a2a9d9d717f06413` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:44.676525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 887/1745] 10.222.0.81 () {58 vars in 1009 bytes} [Mon Sep 19 11:23:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:44.677666 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b372a6-9094-41fc-86b3-6052a2323255 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07cc7bb8955e4255a2a9d9d717f06413)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:44.679646 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b372a6-9094-41fc-86b3-6052a2323255 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:44.858251 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e5b372a6-9094-41fc-86b3-6052a2323255 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07cc7bb8955e4255a2a9d9d717f06413'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:44.883751 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e5b372a6-9094-41fc-86b3-6052a2323255 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07cc7bb8955e4255a2a9d9d717f06413'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:44.884420 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e5b372a6-9094-41fc-86b3-6052a2323255 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07cc7bb8955e4255a2a9d9d717f06413'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:44.885925 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e5b372a6-9094-41fc-86b3-6052a2323255 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07cc7bb8955e4255a2a9d9d717f06413'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:44.917027 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e5b372a6-9094-41fc-86b3-6052a2323255 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07cc7bb8955e4255a2a9d9d717f06413'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:44.947742 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 859/1746] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:23:44 2022] DELETE /identity/v3/users/07cc7bb8955e4255a2a9d9d717f06413 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:45.065981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235e3ee-c87e-4951-8b75-3a52b537fd21 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:45.067441 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-4235e3ee-c87e-4951-8b75-3a52b537fd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:45.370922 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235e3ee-c87e-4951-8b75-3a52b537fd21 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:45.411050 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:45.555575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db229e8-6118-4e0c-a8cb-25e09d782644 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8a9732807d4921a41a6a2d5019e2f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:45.555575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db229e8-6118-4e0c-a8cb-25e09d782644 service swift] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:45.555924 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db229e8-6118-4e0c-a8cb-25e09d782644 service swift] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:45.556075 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db229e8-6118-4e0c-a8cb-25e09d782644 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:45.569380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3db229e8-6118-4e0c-a8cb-25e09d782644 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:45.765178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db229e8-6118-4e0c-a8cb-25e09d782644 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:45.766888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db229e8-6118-4e0c-a8cb-25e09d782644 service swift] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:45.792354 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 888/1747] 10.222.0.81 () {60 vars in 1384 bytes} [Mon Sep 19 11:23:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 726 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:23:45.914965 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b372a6-9094-41fc-86b3-6052a2323255 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:45.917746 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e5b372a6-9094-41fc-86b3-6052a2323255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:46.036068 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b372a6-9094-41fc-86b3-6052a2323255 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:46.045503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f713a-4e25-440f-9a34-81df8b26e6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:46.046976 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f713a-4e25-440f-9a34-81df8b26e6a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:46.047894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f713a-4e25-440f-9a34-81df8b26e6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:46.049144 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f713a-4e25-440f-9a34-81df8b26e6a6 admin admin] PATH_INFO: `/v3/users/9123f95e5acc448a9c4b4514f4a31bdb` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:46.051718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f713a-4e25-440f-9a34-81df8b26e6a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9123f95e5acc448a9c4b4514f4a31bdb)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:46.060957 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f713a-4e25-440f-9a34-81df8b26e6a6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:46.292386 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-501f713a-4e25-440f-9a34-81df8b26e6a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9123f95e5acc448a9c4b4514f4a31bdb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:46.308832 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-501f713a-4e25-440f-9a34-81df8b26e6a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9123f95e5acc448a9c4b4514f4a31bdb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:46.309738 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-501f713a-4e25-440f-9a34-81df8b26e6a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9123f95e5acc448a9c4b4514f4a31bdb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:46.310584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-501f713a-4e25-440f-9a34-81df8b26e6a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9123f95e5acc448a9c4b4514f4a31bdb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:46.327454 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:46.330635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-501f713a-4e25-440f-9a34-81df8b26e6a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9123f95e5acc448a9c4b4514f4a31bdb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:46.373821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 860/1748] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:23:45 2022] DELETE /identity/v3/users/9123f95e5acc448a9c4b4514f4a31bdb => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:46.378319 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c027fcb-c912-4fad-864b-7399b35e8a7f None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:46.378750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c027fcb-c912-4fad-864b-7399b35e8a7f None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:46.379049 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c027fcb-c912-4fad-864b-7399b35e8a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:46.468747 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-2c027fcb-c912-4fad-864b-7399b35e8a7f None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:46.470000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2c027fcb-c912-4fad-864b-7399b35e8a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:46.858115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 889/1749] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:46.918696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f713a-4e25-440f-9a34-81df8b26e6a6 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:46.921054 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-501f713a-4e25-440f-9a34-81df8b26e6a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:47.113482 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f713a-4e25-440f-9a34-81df8b26e6a6 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:47.129520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9931c09-39ec-4f02-8f82-13db077ff855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:47.135596 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9931c09-39ec-4f02-8f82-13db077ff855 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:47.139459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9931c09-39ec-4f02-8f82-13db077ff855 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:47.139902 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9931c09-39ec-4f02-8f82-13db077ff855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:47.172683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9931c09-39ec-4f02-8f82-13db077ff855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:47.179560 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9931c09-39ec-4f02-8f82-13db077ff855 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:47.216877 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 861/1750] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:47.858297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c027fcb-c912-4fad-864b-7399b35e8a7f None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:47.873554 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2c027fcb-c912-4fad-864b-7399b35e8a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:47.908152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c027fcb-c912-4fad-864b-7399b35e8a7f None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:47.935215 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187319b6-34ad-48b9-bdc5-7eac5974d00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:47.939040 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187319b6-34ad-48b9-bdc5-7eac5974d00d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:47.939409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187319b6-34ad-48b9-bdc5-7eac5974d00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:47.939743 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187319b6-34ad-48b9-bdc5-7eac5974d00d admin admin] PATH_INFO: `/v3/projects/7c8ef32903e844e0ba16a7cc9cf9d608` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:47.952532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187319b6-34ad-48b9-bdc5-7eac5974d00d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c8ef32903e844e0ba16a7cc9cf9d608)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:47.961270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187319b6-34ad-48b9-bdc5-7eac5974d00d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:48.175173 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-187319b6-34ad-48b9-bdc5-7eac5974d00d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c8ef32903e844e0ba16a7cc9cf9d608', 'request_id': 'req-187319b6-34ad-48b9-bdc5-7eac5974d00d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:48.207600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-187319b6-34ad-48b9-bdc5-7eac5974d00d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c8ef32903e844e0ba16a7cc9cf9d608', 'request_id': 'req-187319b6-34ad-48b9-bdc5-7eac5974d00d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:48.221676 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:48.251652 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-187319b6-34ad-48b9-bdc5-7eac5974d00d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c8ef32903e844e0ba16a7cc9cf9d608', 'request_id': 'req-187319b6-34ad-48b9-bdc5-7eac5974d00d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:48.263191 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9931c09-39ec-4f02-8f82-13db077ff855 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:48.268538 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d9931c09-39ec-4f02-8f82-13db077ff855 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:48.288498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-187319b6-34ad-48b9-bdc5-7eac5974d00d admin admin] The token cache is being invalidate because project 7c8ef32903e844e0ba16a7cc9cf9d608 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:23:48.288931 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-187319b6-34ad-48b9-bdc5-7eac5974d00d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:48.289785 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-187319b6-34ad-48b9-bdc5-7eac5974d00d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:48.293903 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9931c09-39ec-4f02-8f82-13db077ff855 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:48.296062 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 890/1751] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:23:47 2022] DELETE /identity/v3/projects/7c8ef32903e844e0ba16a7cc9cf9d608 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:48.300005 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187319b6-34ad-48b9-bdc5-7eac5974d00d admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:48.302028 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-187319b6-34ad-48b9-bdc5-7eac5974d00d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:48.513651 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187319b6-34ad-48b9-bdc5-7eac5974d00d admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:48.532612 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93831b73-2244-464b-b2d4-becb439d4ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:48.534809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93831b73-2244-464b-b2d4-becb439d4ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:48.535111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93831b73-2244-464b-b2d4-becb439d4ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:48.535390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93831b73-2244-464b-b2d4-becb439d4ee2 admin admin] PATH_INFO: `/v3/users/a8733f0e3a6242799d1f0bb6a4da01cb` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:48.538551 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93831b73-2244-464b-b2d4-becb439d4ee2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8733f0e3a6242799d1f0bb6a4da01cb)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:48.540577 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d373f2-f3ee-419b-b5a1-0db8689a7f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:48.543878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d373f2-f3ee-419b-b5a1-0db8689a7f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:48.544763 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93831b73-2244-464b-b2d4-becb439d4ee2 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:48.546645 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d373f2-f3ee-419b-b5a1-0db8689a7f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:48.546932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d373f2-f3ee-419b-b5a1-0db8689a7f06 admin admin] PATH_INFO: `/v3/users/e091cd41dab940fdb56ea9179d2298bb` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:48.548895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d373f2-f3ee-419b-b5a1-0db8689a7f06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e091cd41dab940fdb56ea9179d2298bb)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:48.554117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d373f2-f3ee-419b-b5a1-0db8689a7f06 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:48.669961 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-93831b73-2244-464b-b2d4-becb439d4ee2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8733f0e3a6242799d1f0bb6a4da01cb'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:48.671718 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-93831b73-2244-464b-b2d4-becb439d4ee2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8733f0e3a6242799d1f0bb6a4da01cb'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:48.672687 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-93831b73-2244-464b-b2d4-becb439d4ee2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8733f0e3a6242799d1f0bb6a4da01cb'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:48.685511 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-93831b73-2244-464b-b2d4-becb439d4ee2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8733f0e3a6242799d1f0bb6a4da01cb'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:48.686941 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e3d373f2-f3ee-419b-b5a1-0db8689a7f06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e091cd41dab940fdb56ea9179d2298bb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:48.706607 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-93831b73-2244-464b-b2d4-becb439d4ee2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8733f0e3a6242799d1f0bb6a4da01cb'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:48.718752 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e3d373f2-f3ee-419b-b5a1-0db8689a7f06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e091cd41dab940fdb56ea9179d2298bb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:48.719621 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e3d373f2-f3ee-419b-b5a1-0db8689a7f06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e091cd41dab940fdb56ea9179d2298bb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:48.720439 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e3d373f2-f3ee-419b-b5a1-0db8689a7f06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e091cd41dab940fdb56ea9179d2298bb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:48.743731 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 891/1752] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:23:48 2022] DELETE /identity/v3/users/a8733f0e3a6242799d1f0bb6a4da01cb => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:48.745868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a5733-38aa-4280-ab08-28ec9d9f0619 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:48.746348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a5733-38aa-4280-ab08-28ec9d9f0619 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:48.746908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a5733-38aa-4280-ab08-28ec9d9f0619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:48.773048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e3d373f2-f3ee-419b-b5a1-0db8689a7f06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e091cd41dab940fdb56ea9179d2298bb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:48.780895 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-377a5733-38aa-4280-ab08-28ec9d9f0619 None None] MFA Rules not processed for user `9a4a7aca62974db38138a38de399139f`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:48.782346 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-377a5733-38aa-4280-ab08-28ec9d9f0619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:48.825553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 862/1753] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:23:48 2022] DELETE /identity/v3/users/e091cd41dab940fdb56ea9179d2298bb => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:48.828982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e10e89-a344-44cf-8bb2-e7cb6c5090a9 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:48.829269 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e10e89-a344-44cf-8bb2-e7cb6c5090a9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:48.831709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e10e89-a344-44cf-8bb2-e7cb6c5090a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:48.880307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 892/1754] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:48.917227 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:48.954115 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-b6e10e89-a344-44cf-8bb2-e7cb6c5090a9 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:48.971831 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b6e10e89-a344-44cf-8bb2-e7cb6c5090a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:49.201495 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 863/1755] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:49.293617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a5733-38aa-4280-ab08-28ec9d9f0619 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:49.298090 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-377a5733-38aa-4280-ab08-28ec9d9f0619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:49.383384 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:49.539703 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e10e89-a344-44cf-8bb2-e7cb6c5090a9 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:49.541866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b6e10e89-a344-44cf-8bb2-e7cb6c5090a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:49.567438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e10e89-a344-44cf-8bb2-e7cb6c5090a9 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:49.595268 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bd1f7-457a-4dc7-acc5-7b59ca432ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:49.595951 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bd1f7-457a-4dc7-acc5-7b59ca432ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:49.596268 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bd1f7-457a-4dc7-acc5-7b59ca432ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:49.596509 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bd1f7-457a-4dc7-acc5-7b59ca432ac2 admin admin] PATH_INFO: `/v3/users/33b8d13b149c4af48ead3d6650ee7ec5` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:49.598460 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34bd1f7-457a-4dc7-acc5-7b59ca432ac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33b8d13b149c4af48ead3d6650ee7ec5)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:49.606056 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34bd1f7-457a-4dc7-acc5-7b59ca432ac2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:49.637846 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a5733-38aa-4280-ab08-28ec9d9f0619 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:49.865512 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d34bd1f7-457a-4dc7-acc5-7b59ca432ac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33b8d13b149c4af48ead3d6650ee7ec5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:49.881926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d34bd1f7-457a-4dc7-acc5-7b59ca432ac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33b8d13b149c4af48ead3d6650ee7ec5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:49.883064 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d34bd1f7-457a-4dc7-acc5-7b59ca432ac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33b8d13b149c4af48ead3d6650ee7ec5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:49.883891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d34bd1f7-457a-4dc7-acc5-7b59ca432ac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33b8d13b149c4af48ead3d6650ee7ec5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:49.910487 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d34bd1f7-457a-4dc7-acc5-7b59ca432ac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33b8d13b149c4af48ead3d6650ee7ec5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:49.923232 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805bcdf-3b5d-46ab-8f49-7088c27e1fe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:49.928076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805bcdf-3b5d-46ab-8f49-7088c27e1fe3 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:49.930416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805bcdf-3b5d-46ab-8f49-7088c27e1fe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:49.933080 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805bcdf-3b5d-46ab-8f49-7088c27e1fe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:49.941185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e805bcdf-3b5d-46ab-8f49-7088c27e1fe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:49.947612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 864/1756] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:23:49 2022] DELETE /identity/v3/users/33b8d13b149c4af48ead3d6650ee7ec5 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:49.952750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d131c2-eced-4295-b8cd-1a7576692d40 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:49.953265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d131c2-eced-4295-b8cd-1a7576692d40 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:49.953689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d131c2-eced-4295-b8cd-1a7576692d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:50.012750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-62d131c2-eced-4295-b8cd-1a7576692d40 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:50.014585 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-62d131c2-eced-4295-b8cd-1a7576692d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:50.025691 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805bcdf-3b5d-46ab-8f49-7088c27e1fe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:50.036121 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805bcdf-3b5d-46ab-8f49-7088c27e1fe3 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:50.091913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 893/1757] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 801 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:50.203657 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 865/1758] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:50.294194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805bcdf-3b5d-46ab-8f49-7088c27e1fe3 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:50.295632 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e805bcdf-3b5d-46ab-8f49-7088c27e1fe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:50.313435 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805bcdf-3b5d-46ab-8f49-7088c27e1fe3 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:50.378987 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050bd762-dc92-43c4-8f4f-1a33bdc063d8 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:50.379295 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050bd762-dc92-43c4-8f4f-1a33bdc063d8 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:50.379555 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050bd762-dc92-43c4-8f4f-1a33bdc063d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:50.409823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-050bd762-dc92-43c4-8f4f-1a33bdc063d8 None None] MFA Rules not processed for user `65de2e5f67dd4bb6a20cf3934afc8abd`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:50.411088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-050bd762-dc92-43c4-8f4f-1a33bdc063d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:50.448153 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:50.543121 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d8b87-c86a-4666-b480-293fdd67a2bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:50.544233 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d8b87-c86a-4666-b480-293fdd67a2bc service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:50.544576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d8b87-c86a-4666-b480-293fdd67a2bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:50.544909 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d8b87-c86a-4666-b480-293fdd67a2bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:50.585007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d8b87-c86a-4666-b480-293fdd67a2bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:50.591291 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d8b87-c86a-4666-b480-293fdd67a2bc service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:50.625655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 866/1759] 10.222.0.72 () {58 vars in 1015 bytes} [Mon Sep 19 11:23:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:50.635862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 894/1760] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:50.681158 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050bd762-dc92-43c4-8f4f-1a33bdc063d8 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:50.685533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-050bd762-dc92-43c4-8f4f-1a33bdc063d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:50.734386 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050bd762-dc92-43c4-8f4f-1a33bdc063d8 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:50.752244 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259c2c8-baf3-45a1-a2ab-a78f76596d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:50.753110 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259c2c8-baf3-45a1-a2ab-a78f76596d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:50.753705 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259c2c8-baf3-45a1-a2ab-a78f76596d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:50.754128 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259c2c8-baf3-45a1-a2ab-a78f76596d33 admin admin] PATH_INFO: `/v3/projects/9b6e07889e3e42a4877fa56f8712ca94` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:50.756627 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259c2c8-baf3-45a1-a2ab-a78f76596d33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b6e07889e3e42a4877fa56f8712ca94)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:50.797728 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259c2c8-baf3-45a1-a2ab-a78f76596d33 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:50.949093 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4259c2c8-baf3-45a1-a2ab-a78f76596d33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b6e07889e3e42a4877fa56f8712ca94', 'request_id': 'req-4259c2c8-baf3-45a1-a2ab-a78f76596d33'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:50.964452 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4259c2c8-baf3-45a1-a2ab-a78f76596d33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b6e07889e3e42a4877fa56f8712ca94', 'request_id': 'req-4259c2c8-baf3-45a1-a2ab-a78f76596d33'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:50.991988 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4259c2c8-baf3-45a1-a2ab-a78f76596d33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b6e07889e3e42a4877fa56f8712ca94', 'request_id': 'req-4259c2c8-baf3-45a1-a2ab-a78f76596d33'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:51.017292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4259c2c8-baf3-45a1-a2ab-a78f76596d33 admin admin] The token cache is being invalidate because project 9b6e07889e3e42a4877fa56f8712ca94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:23:51.018195 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4259c2c8-baf3-45a1-a2ab-a78f76596d33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:51.018769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4259c2c8-baf3-45a1-a2ab-a78f76596d33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:51.027267 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 867/1761] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:23:50 2022] DELETE /identity/v3/projects/9b6e07889e3e42a4877fa56f8712ca94 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:51.028304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d8b87-c86a-4666-b480-293fdd67a2bc service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:51.044955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d8b87-c86a-4666-b480-293fdd67a2bc service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:51.164232 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfefd3-6a44-4f2a-94d0-867d1a74c182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:51.165724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfefd3-6a44-4f2a-94d0-867d1a74c182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:51.166012 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfefd3-6a44-4f2a-94d0-867d1a74c182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:51.166291 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfefd3-6a44-4f2a-94d0-867d1a74c182 admin admin] PATH_INFO: `/v3/users/f8a69f803cae4b7bb6ccce2943726ff4` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:51.169585 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfefd3-6a44-4f2a-94d0-867d1a74c182 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8a69f803cae4b7bb6ccce2943726ff4)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:51.173352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfefd3-6a44-4f2a-94d0-867d1a74c182 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:51.221554 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:51.270300 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259c2c8-baf3-45a1-a2ab-a78f76596d33 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:51.272462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4259c2c8-baf3-45a1-a2ab-a78f76596d33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:51.323513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-18dfefd3-6a44-4f2a-94d0-867d1a74c182 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8a69f803cae4b7bb6ccce2943726ff4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:51.324286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-18dfefd3-6a44-4f2a-94d0-867d1a74c182 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8a69f803cae4b7bb6ccce2943726ff4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:51.324941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-18dfefd3-6a44-4f2a-94d0-867d1a74c182 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8a69f803cae4b7bb6ccce2943726ff4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:51.341309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-18dfefd3-6a44-4f2a-94d0-867d1a74c182 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8a69f803cae4b7bb6ccce2943726ff4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:51.372740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-18dfefd3-6a44-4f2a-94d0-867d1a74c182 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8a69f803cae4b7bb6ccce2943726ff4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:51.419472 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 895/1762] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:23:51 2022] DELETE /identity/v3/users/f8a69f803cae4b7bb6ccce2943726ff4 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:51.461172 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259c2c8-baf3-45a1-a2ab-a78f76596d33 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:51.657025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f73c3-609e-48ec-b994-080d03392057 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:51.658187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f73c3-609e-48ec-b994-080d03392057 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:51.658533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f73c3-609e-48ec-b994-080d03392057 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:51.658822 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f73c3-609e-48ec-b994-080d03392057 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:51.664763 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b17f73c3-609e-48ec-b994-080d03392057 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:51.685001 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f73c3-609e-48ec-b994-080d03392057 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:51.686988 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f73c3-609e-48ec-b994-080d03392057 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:51.698169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 868/1763] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:51.707280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfefd3-6a44-4f2a-94d0-867d1a74c182 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:51.709624 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-18dfefd3-6a44-4f2a-94d0-867d1a74c182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:51.871447 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfefd3-6a44-4f2a-94d0-867d1a74c182 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:51.887012 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e4310-fc39-498a-980c-3076ca4cb8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:51.887999 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e4310-fc39-498a-980c-3076ca4cb8b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:51.888476 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e4310-fc39-498a-980c-3076ca4cb8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:51.888856 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e4310-fc39-498a-980c-3076ca4cb8b9 admin admin] PATH_INFO: `/v3/projects/34f46c1ef9e5469a8585aabd8cd56bf5` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:51.891626 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e4310-fc39-498a-980c-3076ca4cb8b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34f46c1ef9e5469a8585aabd8cd56bf5)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:51.898592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e4310-fc39-498a-980c-3076ca4cb8b9 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:52.041043 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b83e4310-fc39-498a-980c-3076ca4cb8b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34f46c1ef9e5469a8585aabd8cd56bf5', 'request_id': 'req-b83e4310-fc39-498a-980c-3076ca4cb8b9'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:52.064181 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b83e4310-fc39-498a-980c-3076ca4cb8b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34f46c1ef9e5469a8585aabd8cd56bf5', 'request_id': 'req-b83e4310-fc39-498a-980c-3076ca4cb8b9'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:52.099532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b83e4310-fc39-498a-980c-3076ca4cb8b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34f46c1ef9e5469a8585aabd8cd56bf5', 'request_id': 'req-b83e4310-fc39-498a-980c-3076ca4cb8b9'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:52.165109 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b83e4310-fc39-498a-980c-3076ca4cb8b9 admin admin] The token cache is being invalidate because project 34f46c1ef9e5469a8585aabd8cd56bf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:23:52.165426 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b83e4310-fc39-498a-980c-3076ca4cb8b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:52.174997 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b83e4310-fc39-498a-980c-3076ca4cb8b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:52.183978 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 896/1764] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:23:51 2022] DELETE /identity/v3/projects/34f46c1ef9e5469a8585aabd8cd56bf5 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:52.188101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f73c3-609e-48ec-b994-080d03392057 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:52.211599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f73c3-609e-48ec-b994-080d03392057 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:52.258767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:52.450300 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d872e5-3d82-429c-baaf-7cff4a647b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:52.451056 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d872e5-3d82-429c-baaf-7cff4a647b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:52.451341 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d872e5-3d82-429c-baaf-7cff4a647b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:52.451647 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d872e5-3d82-429c-baaf-7cff4a647b83 admin admin] PATH_INFO: `/v3/users/c170ecbab1fc4916905ecc064dc69cae` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:52.457335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d872e5-3d82-429c-baaf-7cff4a647b83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c170ecbab1fc4916905ecc064dc69cae)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:52.464466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d872e5-3d82-429c-baaf-7cff4a647b83 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:52.591449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-64d872e5-3d82-429c-baaf-7cff4a647b83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c170ecbab1fc4916905ecc064dc69cae'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:52.612319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-64d872e5-3d82-429c-baaf-7cff4a647b83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c170ecbab1fc4916905ecc064dc69cae'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:52.613224 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-64d872e5-3d82-429c-baaf-7cff4a647b83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c170ecbab1fc4916905ecc064dc69cae'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:52.615414 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-64d872e5-3d82-429c-baaf-7cff4a647b83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c170ecbab1fc4916905ecc064dc69cae'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:52.659365 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-64d872e5-3d82-429c-baaf-7cff4a647b83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c170ecbab1fc4916905ecc064dc69cae'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:52.699132 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 869/1765] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:23:52 2022] DELETE /identity/v3/users/c170ecbab1fc4916905ecc064dc69cae => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:52.753103 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e4310-fc39-498a-980c-3076ca4cb8b9 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:52.757592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b83e4310-fc39-498a-980c-3076ca4cb8b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:52.782561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e4310-fc39-498a-980c-3076ca4cb8b9 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:52.850622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:53.011509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ec699-5e5c-4b0d-b43f-55071a45bb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:53.013748 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ec699-5e5c-4b0d-b43f-55071a45bb4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:53.014962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ec699-5e5c-4b0d-b43f-55071a45bb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:53.016703 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ec699-5e5c-4b0d-b43f-55071a45bb4d admin admin] PATH_INFO: `/v3/projects/6a02a26bfc664cb4a9d0cbbd754647c7` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:53.022705 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ec699-5e5c-4b0d-b43f-55071a45bb4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a02a26bfc664cb4a9d0cbbd754647c7)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:53.028991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ec699-5e5c-4b0d-b43f-55071a45bb4d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:53.162749 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c53ec699-5e5c-4b0d-b43f-55071a45bb4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a02a26bfc664cb4a9d0cbbd754647c7', 'request_id': 'req-c53ec699-5e5c-4b0d-b43f-55071a45bb4d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:53.180986 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c53ec699-5e5c-4b0d-b43f-55071a45bb4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a02a26bfc664cb4a9d0cbbd754647c7', 'request_id': 'req-c53ec699-5e5c-4b0d-b43f-55071a45bb4d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:53.201514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c53ec699-5e5c-4b0d-b43f-55071a45bb4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a02a26bfc664cb4a9d0cbbd754647c7', 'request_id': 'req-c53ec699-5e5c-4b0d-b43f-55071a45bb4d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:53.231337 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c53ec699-5e5c-4b0d-b43f-55071a45bb4d admin admin] The token cache is being invalidate because project 6a02a26bfc664cb4a9d0cbbd754647c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:23:53.233850 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c53ec699-5e5c-4b0d-b43f-55071a45bb4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:53.235565 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c53ec699-5e5c-4b0d-b43f-55071a45bb4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:53.238758 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 897/1766] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:23:52 2022] DELETE /identity/v3/projects/6a02a26bfc664cb4a9d0cbbd754647c7 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:53.243461 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb5a05-870c-41b9-8232-22c28284e1e2 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:53.244147 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb5a05-870c-41b9-8232-22c28284e1e2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:53.244791 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb5a05-870c-41b9-8232-22c28284e1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:53.286973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ec699-5e5c-4b0d-b43f-55071a45bb4d admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:53.288617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c53ec699-5e5c-4b0d-b43f-55071a45bb4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:53.341967 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-3eeb5a05-870c-41b9-8232-22c28284e1e2 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:53.344889 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3eeb5a05-870c-41b9-8232-22c28284e1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:53.540502 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ec699-5e5c-4b0d-b43f-55071a45bb4d admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:53.553235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 870/1767] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:53.559311 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab50cde-b8e7-49c1-a08d-7cc693637c4d None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:53.560223 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab50cde-b8e7-49c1-a08d-7cc693637c4d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:53.560639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b02a78-1ba4-4417-9d6b-67daf3afc6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:53.560839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab50cde-b8e7-49c1-a08d-7cc693637c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:53.561499 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b02a78-1ba4-4417-9d6b-67daf3afc6b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:53.561884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b02a78-1ba4-4417-9d6b-67daf3afc6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:53.572915 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b02a78-1ba4-4417-9d6b-67daf3afc6b9 admin admin] PATH_INFO: `/v3/projects/7f876df472184339907077a54d688fc3` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:53.576004 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b02a78-1ba4-4417-9d6b-67daf3afc6b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f876df472184339907077a54d688fc3)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:53.578424 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b02a78-1ba4-4417-9d6b-67daf3afc6b9 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:53.610003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-7ab50cde-b8e7-49c1-a08d-7cc693637c4d None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:53.612044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7ab50cde-b8e7-49c1-a08d-7cc693637c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:53.633800 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 871/1768] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:53.638914 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab50cde-b8e7-49c1-a08d-7cc693637c4d None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:53.667667 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab50cde-b8e7-49c1-a08d-7cc693637c4d None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:53.692911 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b78ea2-11c1-46ab-af83-6f4318ea94f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:53.694244 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b78ea2-11c1-46ab-af83-6f4318ea94f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:53.694742 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b78ea2-11c1-46ab-af83-6f4318ea94f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:53.695161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b78ea2-11c1-46ab-af83-6f4318ea94f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:53.696026 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b78ea2-11c1-46ab-af83-6f4318ea94f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:23:53.696315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b78ea2-11c1-46ab-af83-6f4318ea94f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:53.698819 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b78ea2-11c1-46ab-af83-6f4318ea94f0 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:53.733071 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 872/1769] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:23:53.738125 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b78ea2-11c1-46ab-af83-6f4318ea94f0 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:53.764320 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b78ea2-11c1-46ab-af83-6f4318ea94f0 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:53.769559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-98b02a78-1ba4-4417-9d6b-67daf3afc6b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f876df472184339907077a54d688fc3', 'request_id': 'req-98b02a78-1ba4-4417-9d6b-67daf3afc6b9'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:53.779568 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:53.791384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:53.799167 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-98b02a78-1ba4-4417-9d6b-67daf3afc6b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f876df472184339907077a54d688fc3', 'request_id': 'req-98b02a78-1ba4-4417-9d6b-67daf3afc6b9'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:53.813511 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-98b02a78-1ba4-4417-9d6b-67daf3afc6b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f876df472184339907077a54d688fc3', 'request_id': 'req-98b02a78-1ba4-4417-9d6b-67daf3afc6b9'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:53.844709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-98b02a78-1ba4-4417-9d6b-67daf3afc6b9 admin admin] The token cache is being invalidate because project 7f876df472184339907077a54d688fc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:23:53.845006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-98b02a78-1ba4-4417-9d6b-67daf3afc6b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:53.849620 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-98b02a78-1ba4-4417-9d6b-67daf3afc6b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:53.851270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 898/1770] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:23:53 2022] DELETE /identity/v3/projects/7f876df472184339907077a54d688fc3 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:53.856629 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a15e77-45b7-4b7b-938c-be14a53a4127 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:53.856910 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a15e77-45b7-4b7b-938c-be14a53a4127 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:53.857189 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a15e77-45b7-4b7b-938c-be14a53a4127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:53.894648 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-92a15e77-45b7-4b7b-938c-be14a53a4127 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:53.895937 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-92a15e77-45b7-4b7b-938c-be14a53a4127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:53.982578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c47c2-0fc7-471f-aa00-8459c6481a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:53.982578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c47c2-0fc7-471f-aa00-8459c6481a92 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:53.982578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c47c2-0fc7-471f-aa00-8459c6481a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:53.982578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c47c2-0fc7-471f-aa00-8459c6481a92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:53.983482 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c47c2-0fc7-471f-aa00-8459c6481a92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:53.992118 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c47c2-0fc7-471f-aa00-8459c6481a92 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:54.050913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 899/1771] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:23:54.053806 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ef540-2172-4f6c-b360-98a3a37173a4 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:54.054095 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ef540-2172-4f6c-b360-98a3a37173a4 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:54.054377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ef540-2172-4f6c-b360-98a3a37173a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:54.090635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 873/1772] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:53 2022] POST /identity/v3/projects => generated 371 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:23:54.091648 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c47c2-0fc7-471f-aa00-8459c6481a92 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:54.094690 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c16c47c2-0fc7-471f-aa00-8459c6481a92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:54.108568 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-a23ef540-2172-4f6c-b360-98a3a37173a4 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:54.109720 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a23ef540-2172-4f6c-b360-98a3a37173a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:54.292518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c47c2-0fc7-471f-aa00-8459c6481a92 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:54.300276 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ec988-7d51-45eb-9471-88098490dc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:54.301112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ec988-7d51-45eb-9471-88098490dc78 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:54.301645 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ec988-7d51-45eb-9471-88098490dc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:54.302058 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ec988-7d51-45eb-9471-88098490dc78 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:54.302818 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ec988-7d51-45eb-9471-88098490dc78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:54.305578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ec988-7d51-45eb-9471-88098490dc78 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:54.324913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 900/1773] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:54.329074 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ef540-2172-4f6c-b360-98a3a37173a4 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:54.350316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ef540-2172-4f6c-b360-98a3a37173a4 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:54.359678 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 874/1774] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:54 2022] POST /identity/v3/users => generated 286 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:54.361692 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ec988-7d51-45eb-9471-88098490dc78 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:54.364106 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7930bc-f2bc-4fde-a764-5f8d5d8829bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:54.364849 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7930bc-f2bc-4fde-a764-5f8d5d8829bc admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:54.365196 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7930bc-f2bc-4fde-a764-5f8d5d8829bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:54.365547 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7930bc-f2bc-4fde-a764-5f8d5d8829bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:54.366210 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7930bc-f2bc-4fde-a764-5f8d5d8829bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:23:54.366550 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7930bc-f2bc-4fde-a764-5f8d5d8829bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:54.371462 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7930bc-f2bc-4fde-a764-5f8d5d8829bc admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:54.395488 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ec988-7d51-45eb-9471-88098490dc78 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:54.409611 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca45cea-2454-4edc-af56-749a7fdbe479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:54.409994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 901/1775] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:23:54.413181 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca45cea-2454-4edc-af56-749a7fdbe479 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:54.414136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca45cea-2454-4edc-af56-749a7fdbe479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:54.414527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca45cea-2454-4edc-af56-749a7fdbe479 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:54.414865 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7930bc-f2bc-4fde-a764-5f8d5d8829bc admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:54.417190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca45cea-2454-4edc-af56-749a7fdbe479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:54.422024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca45cea-2454-4edc-af56-749a7fdbe479 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:54.444847 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7930bc-f2bc-4fde-a764-5f8d5d8829bc admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:54.456126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 875/1776] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:54.458009 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca45cea-2454-4edc-af56-749a7fdbe479 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:54.470123 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9697b4a6-9c00-4787-a202-a15ef587e549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:54.470844 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697b4a6-9c00-4787-a202-a15ef587e549 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:54.471193 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697b4a6-9c00-4787-a202-a15ef587e549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:54.471545 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697b4a6-9c00-4787-a202-a15ef587e549 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:54.472280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9697b4a6-9c00-4787-a202-a15ef587e549 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:54.479866 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9697b4a6-9c00-4787-a202-a15ef587e549 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:54.494096 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca45cea-2454-4edc-af56-749a7fdbe479 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:54.506859 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c9ee7-32f2-44a4-a6e3-ab9883db0dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:54.507723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c9ee7-32f2-44a4-a6e3-ab9883db0dbf admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:54.508249 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c9ee7-32f2-44a4-a6e3-ab9883db0dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:54.508647 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c9ee7-32f2-44a4-a6e3-ab9883db0dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:54.510417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c9ee7-32f2-44a4-a6e3-ab9883db0dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:54.512435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c9ee7-32f2-44a4-a6e3-ab9883db0dbf admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:54.567985 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 876/1777] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:54.569077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c9ee7-32f2-44a4-a6e3-ab9883db0dbf admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:54.602510 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c9ee7-32f2-44a4-a6e3-ab9883db0dbf admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:54.608582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 902/1778] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:54 2022] POST /identity/v3/projects => generated 405 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:54.615471 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9697b4a6-9c00-4787-a202-a15ef587e549 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:54.619160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499e847-be82-486f-b898-811ff5eb62d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:54.620169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499e847-be82-486f-b898-811ff5eb62d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:54.620700 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499e847-be82-486f-b898-811ff5eb62d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:54.621256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499e847-be82-486f-b898-811ff5eb62d2 admin admin] PATH_INFO: `/v3/projects/cdf9ae08ece14e2baf56fdf8b4ea2c39/users/833bfed6f72b419281678aca7c28b2e4/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:54.687748 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499e847-be82-486f-b898-811ff5eb62d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdf9ae08ece14e2baf56fdf8b4ea2c39, user_id=833bfed6f72b419281678aca7c28b2e4, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:54.704941 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499e847-be82-486f-b898-811ff5eb62d2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:54.761567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:54.775285 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 877/1779] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:54 2022] PUT /identity/v3/projects/cdf9ae08ece14e2baf56fdf8b4ea2c39/users/833bfed6f72b419281678aca7c28b2e4/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:54.781081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b1c5c-bfa1-49af-a9bc-78fe6c43482d None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:54.781377 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b1c5c-bfa1-49af-a9bc-78fe6c43482d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:54.796533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b1c5c-bfa1-49af-a9bc-78fe6c43482d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:54.805599 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9697b4a6-9c00-4787-a202-a15ef587e549 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:54.872355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d89b1c5c-bfa1-49af-a9bc-78fe6c43482d None None] MFA Rules not processed for user `a3097d9c130746ca840bf85f33bc006b`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:54.873995 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d89b1c5c-bfa1-49af-a9bc-78fe6c43482d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:54.924590 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed58380-8afa-4ba2-952e-04b66946b508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:54.925407 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed58380-8afa-4ba2-952e-04b66946b508 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:54.925812 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed58380-8afa-4ba2-952e-04b66946b508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:54.926191 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed58380-8afa-4ba2-952e-04b66946b508 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:54.927044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed58380-8afa-4ba2-952e-04b66946b508 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:54.929415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed58380-8afa-4ba2-952e-04b66946b508 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:54.976950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:54.998856 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 903/1780] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:54 2022] POST /identity/v3/users => generated 303 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:55.000448 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf69bd-6990-4802-8a12-13fe825934ab None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:55.000804 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf69bd-6990-4802-8a12-13fe825934ab None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:55.001272 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf69bd-6990-4802-8a12-13fe825934ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:55.041107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-f9bf69bd-6990-4802-8a12-13fe825934ab None None] MFA Rules not processed for user `833bfed6f72b419281678aca7c28b2e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:55.044406 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f9bf69bd-6990-4802-8a12-13fe825934ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:55.093078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 878/1781] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:55.100590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b1c5c-bfa1-49af-a9bc-78fe6c43482d None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:55.144051 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b1c5c-bfa1-49af-a9bc-78fe6c43482d None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:55.164192 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049d82a-2cbf-42b5-804e-dc6fd92ed559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:55.166376 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049d82a-2cbf-42b5-804e-dc6fd92ed559 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:55.166671 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049d82a-2cbf-42b5-804e-dc6fd92ed559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:55.166954 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049d82a-2cbf-42b5-804e-dc6fd92ed559 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:55.169149 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049d82a-2cbf-42b5-804e-dc6fd92ed559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:55.174670 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049d82a-2cbf-42b5-804e-dc6fd92ed559 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:55.229359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 879/1782] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:55.249108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049d82a-2cbf-42b5-804e-dc6fd92ed559 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:55.260049 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 904/1783] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:55.268784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf69bd-6990-4802-8a12-13fe825934ab None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:55.286150 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049d82a-2cbf-42b5-804e-dc6fd92ed559 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:55.307490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3350b4-d741-430c-836e-5be93e2ea6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:55.308159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3350b4-d741-430c-836e-5be93e2ea6a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:55.308445 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3350b4-d741-430c-836e-5be93e2ea6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:55.308970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3350b4-d741-430c-836e-5be93e2ea6a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:55.309629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3350b4-d741-430c-836e-5be93e2ea6a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:55.317149 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3350b4-d741-430c-836e-5be93e2ea6a6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:55.320064 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf69bd-6990-4802-8a12-13fe825934ab None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:55.349090 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5a226-97da-4560-95f4-ae8cac7b794c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:55.350181 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5a226-97da-4560-95f4-ae8cac7b794c admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:55.350181 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5a226-97da-4560-95f4-ae8cac7b794c admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:55.350523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5a226-97da-4560-95f4-ae8cac7b794c admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:55.351247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5a226-97da-4560-95f4-ae8cac7b794c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:55.365363 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5a226-97da-4560-95f4-ae8cac7b794c admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:55.373162 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 880/1784] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:55.374277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3350b4-d741-430c-836e-5be93e2ea6a6 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:55.406701 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3350b4-d741-430c-836e-5be93e2ea6a6 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:55.423419 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8de2e3-0317-4f46-9c47-940b45102a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:55.424328 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8de2e3-0317-4f46-9c47-940b45102a65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:55.424711 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8de2e3-0317-4f46-9c47-940b45102a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:55.425049 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8de2e3-0317-4f46-9c47-940b45102a65 admin admin] PATH_INFO: `/v3/projects/f98ea508636e4fe49dc5fb553c10a79e/users/87c4421d1dac42b796be3d32e0a747db/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:55.449173 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 905/1785] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:55 2022] POST /identity/v3/projects => generated 371 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:55.451200 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5a226-97da-4560-95f4-ae8cac7b794c admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:55.472523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8de2e3-0317-4f46-9c47-940b45102a65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f98ea508636e4fe49dc5fb553c10a79e, user_id=87c4421d1dac42b796be3d32e0a747db, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:55.475897 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8de2e3-0317-4f46-9c47-940b45102a65 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:55.504334 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 881/1786] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:55 2022] PUT /identity/v3/projects/f98ea508636e4fe49dc5fb553c10a79e/users/87c4421d1dac42b796be3d32e0a747db/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:55.521533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e53f4-77f0-4679-949d-01390afdee84 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:55.521829 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e53f4-77f0-4679-949d-01390afdee84 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:55.522109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e53f4-77f0-4679-949d-01390afdee84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:55.585998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-607e53f4-77f0-4679-949d-01390afdee84 None None] MFA Rules not processed for user `87c4421d1dac42b796be3d32e0a747db`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:55.588700 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-607e53f4-77f0-4679-949d-01390afdee84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:55.644610 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:55.688532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5a226-97da-4560-95f4-ae8cac7b794c admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:55.805748 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 882/1787] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:55 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:55.808856 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e53f4-77f0-4679-949d-01390afdee84 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:55.930978 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f376ac-5635-4638-b1fa-11e776dc36e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:55.932345 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f376ac-5635-4638-b1fa-11e776dc36e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:55.932630 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f376ac-5635-4638-b1fa-11e776dc36e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:55.932904 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f376ac-5635-4638-b1fa-11e776dc36e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:55.933546 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f376ac-5635-4638-b1fa-11e776dc36e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:55.935502 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f376ac-5635-4638-b1fa-11e776dc36e4 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:55.975799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 906/1788] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:55 2022] POST /identity/v3/users => generated 292 bytes in 525 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:55.980563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f376ac-5635-4638-b1fa-11e776dc36e4 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:55.997412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e53f4-77f0-4679-949d-01390afdee84 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:56.004083 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f376ac-5635-4638-b1fa-11e776dc36e4 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:56.007698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5c467-3e0f-47ca-a229-5fb83f142210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:56.008359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5c467-3e0f-47ca-a229-5fb83f142210 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:56.008643 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5c467-3e0f-47ca-a229-5fb83f142210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:56.008938 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5c467-3e0f-47ca-a229-5fb83f142210 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:56.010196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5c467-3e0f-47ca-a229-5fb83f142210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:56.012125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dfc17-ac04-421d-bd55-dc46a399a7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:56.012269 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5c467-3e0f-47ca-a229-5fb83f142210 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:56.012814 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dfc17-ac04-421d-bd55-dc46a399a7e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:56.013096 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dfc17-ac04-421d-bd55-dc46a399a7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:56.013404 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dfc17-ac04-421d-bd55-dc46a399a7e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:56.014100 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dfc17-ac04-421d-bd55-dc46a399a7e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:56.020358 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dfc17-ac04-421d-bd55-dc46a399a7e1 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:56.043825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 907/1789] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:23:56.043825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dfc17-ac04-421d-bd55-dc46a399a7e1 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:56.070851 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dfc17-ac04-421d-bd55-dc46a399a7e1 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:56.082308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb648a6c-a563-4129-89a9-0db964aa2a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:56.083072 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb648a6c-a563-4129-89a9-0db964aa2a5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:56.083422 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb648a6c-a563-4129-89a9-0db964aa2a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:56.084430 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb648a6c-a563-4129-89a9-0db964aa2a5d admin admin] PATH_INFO: `/v3/projects/f1b67b20446b4ceebe55e9aab09f8655/users/0fcec5c32aaf4d679db48a496847eaf8/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:56.084563 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 883/1790] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:55 2022] POST /identity/v3/projects => generated 407 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:56.085438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5c467-3e0f-47ca-a229-5fb83f142210 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:56.125466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:56.133142 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb648a6c-a563-4129-89a9-0db964aa2a5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b67b20446b4ceebe55e9aab09f8655, user_id=0fcec5c32aaf4d679db48a496847eaf8, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:56.136172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb648a6c-a563-4129-89a9-0db964aa2a5d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:56.164806 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 908/1791] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:56 2022] PUT /identity/v3/projects/f1b67b20446b4ceebe55e9aab09f8655/users/0fcec5c32aaf4d679db48a496847eaf8/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:56.166496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb648a6c-a563-4129-89a9-0db964aa2a5d admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:56.173722 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-eb648a6c-a563-4129-89a9-0db964aa2a5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:56.245682 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5c467-3e0f-47ca-a229-5fb83f142210 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:56.330461 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb648a6c-a563-4129-89a9-0db964aa2a5d admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:56.352192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c716cc9-ee2c-465a-939f-20d31063f2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:56.353037 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c716cc9-ee2c-465a-939f-20d31063f2c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:56.353492 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c716cc9-ee2c-465a-939f-20d31063f2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:56.353898 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c716cc9-ee2c-465a-939f-20d31063f2c3 admin admin] PATH_INFO: `/v3/projects/dcb15b699d5f4db990b0f9fd451eb470` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:56.356535 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c716cc9-ee2c-465a-939f-20d31063f2c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcb15b699d5f4db990b0f9fd451eb470)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:56.363257 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c716cc9-ee2c-465a-939f-20d31063f2c3 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:56.381741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea7d8c-9876-47fc-bbcf-d7506b0bcdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:56.382555 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea7d8c-9876-47fc-bbcf-d7506b0bcdb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:56.382991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea7d8c-9876-47fc-bbcf-d7506b0bcdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:56.384003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea7d8c-9876-47fc-bbcf-d7506b0bcdb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:56.385663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea7d8c-9876-47fc-bbcf-d7506b0bcdb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:56.388858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea7d8c-9876-47fc-bbcf-d7506b0bcdb8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:56.438220 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 884/1792] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:56 2022] POST /identity/v3/users => generated 310 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:56.439330 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea7d8c-9876-47fc-bbcf-d7506b0bcdb8 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:56.466494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea7d8c-9876-47fc-bbcf-d7506b0bcdb8 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:56.483294 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-7c716cc9-ee2c-465a-939f-20d31063f2c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcb15b699d5f4db990b0f9fd451eb470', 'request_id': 'req-7c716cc9-ee2c-465a-939f-20d31063f2c3'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:56.502348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-7c716cc9-ee2c-465a-939f-20d31063f2c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcb15b699d5f4db990b0f9fd451eb470', 'request_id': 'req-7c716cc9-ee2c-465a-939f-20d31063f2c3'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:56.509964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:56.527367 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-7c716cc9-ee2c-465a-939f-20d31063f2c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcb15b699d5f4db990b0f9fd451eb470', 'request_id': 'req-7c716cc9-ee2c-465a-939f-20d31063f2c3'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:56.559999 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-7c716cc9-ee2c-465a-939f-20d31063f2c3 admin admin] The token cache is being invalidate because project dcb15b699d5f4db990b0f9fd451eb470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:23:56.562056 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-7c716cc9-ee2c-465a-939f-20d31063f2c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:56.563558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-7c716cc9-ee2c-465a-939f-20d31063f2c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:56.569025 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 909/1793] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:23:56 2022] DELETE /identity/v3/projects/dcb15b699d5f4db990b0f9fd451eb470 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:56.578709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c716cc9-ee2c-465a-939f-20d31063f2c3 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:56.583521 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-7c716cc9-ee2c-465a-939f-20d31063f2c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:56.678187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55e4ac-5879-4510-ad54-fdb56dacdd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:56.678825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55e4ac-5879-4510-ad54-fdb56dacdd40 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:56.679113 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55e4ac-5879-4510-ad54-fdb56dacdd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:56.679391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55e4ac-5879-4510-ad54-fdb56dacdd40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:56.680092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55e4ac-5879-4510-ad54-fdb56dacdd40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:56.681777 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55e4ac-5879-4510-ad54-fdb56dacdd40 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:56.703288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 885/1794] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:56.722022 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55e4ac-5879-4510-ad54-fdb56dacdd40 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:56.724841 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-aa55e4ac-5879-4510-ad54-fdb56dacdd40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:56.753508 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55e4ac-5879-4510-ad54-fdb56dacdd40 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:56.767310 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f4d39-03d3-46f7-b5f9-4e75c691c3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:56.768119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f4d39-03d3-46f7-b5f9-4e75c691c3c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:56.768405 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f4d39-03d3-46f7-b5f9-4e75c691c3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:56.768686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f4d39-03d3-46f7-b5f9-4e75c691c3c2 admin admin] PATH_INFO: `/v3/projects/3daf7ecec0a34c6585b9e0a950b5f5ca` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:56.771247 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f4d39-03d3-46f7-b5f9-4e75c691c3c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3daf7ecec0a34c6585b9e0a950b5f5ca)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:56.773172 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f4d39-03d3-46f7-b5f9-4e75c691c3c2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:56.828684 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c716cc9-ee2c-465a-939f-20d31063f2c3 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:56.837757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25abfea-d32d-45dd-b68b-ed2c6e855837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:56.839745 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25abfea-d32d-45dd-b68b-ed2c6e855837 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:56.840348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25abfea-d32d-45dd-b68b-ed2c6e855837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:56.840849 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25abfea-d32d-45dd-b68b-ed2c6e855837 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:56.842002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25abfea-d32d-45dd-b68b-ed2c6e855837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:56.843913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25abfea-d32d-45dd-b68b-ed2c6e855837 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:56.867177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 910/1795] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:56.868658 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25abfea-d32d-45dd-b68b-ed2c6e855837 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:56.872464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c25abfea-d32d-45dd-b68b-ed2c6e855837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:56.903166 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25abfea-d32d-45dd-b68b-ed2c6e855837 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:56.916830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:56.934546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9c3f4d39-03d3-46f7-b5f9-4e75c691c3c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3daf7ecec0a34c6585b9e0a950b5f5ca', 'request_id': 'req-9c3f4d39-03d3-46f7-b5f9-4e75c691c3c2'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:56.967304 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9c3f4d39-03d3-46f7-b5f9-4e75c691c3c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3daf7ecec0a34c6585b9e0a950b5f5ca', 'request_id': 'req-9c3f4d39-03d3-46f7-b5f9-4e75c691c3c2'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:56.988398 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9c3f4d39-03d3-46f7-b5f9-4e75c691c3c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3daf7ecec0a34c6585b9e0a950b5f5ca', 'request_id': 'req-9c3f4d39-03d3-46f7-b5f9-4e75c691c3c2'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:57.041146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9c3f4d39-03d3-46f7-b5f9-4e75c691c3c2 admin admin] The token cache is being invalidate because project 3daf7ecec0a34c6585b9e0a950b5f5ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:23:57.041357 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9c3f4d39-03d3-46f7-b5f9-4e75c691c3c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:57.044191 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9c3f4d39-03d3-46f7-b5f9-4e75c691c3c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:57.049170 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 886/1796] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:23:56 2022] DELETE /identity/v3/projects/3daf7ecec0a34c6585b9e0a950b5f5ca => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:57.049170 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f4d39-03d3-46f7-b5f9-4e75c691c3c2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:57.049510 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9c3f4d39-03d3-46f7-b5f9-4e75c691c3c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:57.106037 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1d5e8-6a41-4107-bd09-01121d895a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:57.106963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1d5e8-6a41-4107-bd09-01121d895a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:57.107369 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1d5e8-6a41-4107-bd09-01121d895a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:57.107840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1d5e8-6a41-4107-bd09-01121d895a4c admin admin] PATH_INFO: `/v3/users/712a3d8e605c447d8d12c60489886304` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:57.109360 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1d5e8-6a41-4107-bd09-01121d895a4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=712a3d8e605c447d8d12c60489886304)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:57.111781 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1d5e8-6a41-4107-bd09-01121d895a4c admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:57.244595 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c2a1d5e8-6a41-4107-bd09-01121d895a4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '712a3d8e605c447d8d12c60489886304'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:57.245936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c2a1d5e8-6a41-4107-bd09-01121d895a4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '712a3d8e605c447d8d12c60489886304'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:57.248096 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c2a1d5e8-6a41-4107-bd09-01121d895a4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '712a3d8e605c447d8d12c60489886304'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:57.266814 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f4d39-03d3-46f7-b5f9-4e75c691c3c2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:57.267999 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c2a1d5e8-6a41-4107-bd09-01121d895a4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '712a3d8e605c447d8d12c60489886304'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:57.278570 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60441b-32b9-4349-a135-b5207e0a5eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:57.281266 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60441b-32b9-4349-a135-b5207e0a5eac admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:57.281444 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60441b-32b9-4349-a135-b5207e0a5eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:57.281760 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60441b-32b9-4349-a135-b5207e0a5eac admin admin] PATH_INFO: `/v3/projects/f1b67b20446b4ceebe55e9aab09f8655/users/0fcec5c32aaf4d679db48a496847eaf8/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:57.287726 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60441b-32b9-4349-a135-b5207e0a5eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b67b20446b4ceebe55e9aab09f8655, user_id=0fcec5c32aaf4d679db48a496847eaf8, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:57.297587 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60441b-32b9-4349-a135-b5207e0a5eac admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:57.304018 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c2a1d5e8-6a41-4107-bd09-01121d895a4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '712a3d8e605c447d8d12c60489886304'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:57.318789 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:57.329255 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 911/1797] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:23:56 2022] DELETE /identity/v3/users/712a3d8e605c447d8d12c60489886304 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:57.330546 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1d5e8-6a41-4107-bd09-01121d895a4c admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:57.332225 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c2a1d5e8-6a41-4107-bd09-01121d895a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:57.341777 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 887/1798] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:57 2022] PUT /identity/v3/projects/f1b67b20446b4ceebe55e9aab09f8655/users/0fcec5c32aaf4d679db48a496847eaf8/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:57.343259 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c5fd5-99ba-4348-9c28-e5314aa6fff6 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:57.343552 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c5fd5-99ba-4348-9c28-e5314aa6fff6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:57.343852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c5fd5-99ba-4348-9c28-e5314aa6fff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:57.411732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-022c5fd5-99ba-4348-9c28-e5314aa6fff6 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:57.413184 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-022c5fd5-99ba-4348-9c28-e5314aa6fff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:57.484583 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1d5e8-6a41-4107-bd09-01121d895a4c admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:57.494042 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d69038-2dab-4cfa-a3b9-8ef6b40bed35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:57.495603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d69038-2dab-4cfa-a3b9-8ef6b40bed35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:57.495946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d69038-2dab-4cfa-a3b9-8ef6b40bed35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:57.496606 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d69038-2dab-4cfa-a3b9-8ef6b40bed35 admin admin] PATH_INFO: `/v3/projects/6f7098dc7673435f8ed3b4c2896e494e/users/318f2d1fb49e4b2dac41b61b1108f32b/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:57.568605 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d69038-2dab-4cfa-a3b9-8ef6b40bed35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f7098dc7673435f8ed3b4c2896e494e, user_id=318f2d1fb49e4b2dac41b61b1108f32b, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:57.579583 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d69038-2dab-4cfa-a3b9-8ef6b40bed35 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:57.604996 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 912/1799] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:57 2022] PUT /identity/v3/projects/6f7098dc7673435f8ed3b4c2896e494e/users/318f2d1fb49e4b2dac41b61b1108f32b/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:57.608168 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041707e-11be-4a9b-b247-297dcda002a7 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:57.608645 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041707e-11be-4a9b-b247-297dcda002a7 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:57.609515 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041707e-11be-4a9b-b247-297dcda002a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:57.630071 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 888/1800] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:57.641285 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c5fd5-99ba-4348-9c28-e5314aa6fff6 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:57.675271 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c5fd5-99ba-4348-9c28-e5314aa6fff6 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:57.683156 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-a041707e-11be-4a9b-b247-297dcda002a7 None None] MFA Rules not processed for user `0fcec5c32aaf4d679db48a496847eaf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:57.687702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a041707e-11be-4a9b-b247-297dcda002a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:57.809034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b8935-b6c4-4ac7-8b54-0b48efaa59e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:57.810166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b8935-b6c4-4ac7-8b54-0b48efaa59e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:57.810681 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b8935-b6c4-4ac7-8b54-0b48efaa59e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:57.811196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b8935-b6c4-4ac7-8b54-0b48efaa59e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:57.811999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b8935-b6c4-4ac7-8b54-0b48efaa59e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:57.813838 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b8935-b6c4-4ac7-8b54-0b48efaa59e3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:57.839827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:57.857434 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 889/1801] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:57.859109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28a565-0cd0-4c54-b664-7b13ea7eb095 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:57.859864 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28a565-0cd0-4c54-b664-7b13ea7eb095 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:57.861011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28a565-0cd0-4c54-b664-7b13ea7eb095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:57.906279 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-2e28a565-0cd0-4c54-b664-7b13ea7eb095 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:57.908235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2e28a565-0cd0-4c54-b664-7b13ea7eb095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:57.932359 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 913/1802] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:57.932359 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041707e-11be-4a9b-b247-297dcda002a7 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:57.932672 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 890/1803] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:57.943391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28a565-0cd0-4c54-b664-7b13ea7eb095 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:57.955858 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041707e-11be-4a9b-b247-297dcda002a7 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:57.957540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28a565-0cd0-4c54-b664-7b13ea7eb095 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:57.966237 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ccc06-3712-410c-b305-f25a25f718e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:57.967401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ccc06-3712-410c-b305-f25a25f718e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:57.967734 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ccc06-3712-410c-b305-f25a25f718e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:57.968147 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ccc06-3712-410c-b305-f25a25f718e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:57.969025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ccc06-3712-410c-b305-f25a25f718e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:23:57.969403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ccc06-3712-410c-b305-f25a25f718e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:57.969960 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601a892-7d62-41d7-b880-8e7bea8395fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:57.970800 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601a892-7d62-41d7-b880-8e7bea8395fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:57.971230 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601a892-7d62-41d7-b880-8e7bea8395fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:57.971496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ccc06-3712-410c-b305-f25a25f718e2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:57.971846 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601a892-7d62-41d7-b880-8e7bea8395fd admin admin] PATH_INFO: `/v3/projects/6f7098dc7673435f8ed3b4c2896e494e/users/318f2d1fb49e4b2dac41b61b1108f32b/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:57.974780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601a892-7d62-41d7-b880-8e7bea8395fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f7098dc7673435f8ed3b4c2896e494e, user_id=318f2d1fb49e4b2dac41b61b1108f32b, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:57.977841 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601a892-7d62-41d7-b880-8e7bea8395fd admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:57.990105 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 914/1804] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:57 2022] PUT /identity/v3/projects/6f7098dc7673435f8ed3b4c2896e494e/users/318f2d1fb49e4b2dac41b61b1108f32b/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:57.991116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601a892-7d62-41d7-b880-8e7bea8395fd admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:57.993075 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-0601a892-7d62-41d7-b880-8e7bea8395fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:58.006743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 891/1805] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:23:58.008362 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1943304-83ab-47fe-bb87-03d1dfc90fa5 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:58.008648 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1943304-83ab-47fe-bb87-03d1dfc90fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:58.008929 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1943304-83ab-47fe-bb87-03d1dfc90fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:58.080199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f1943304-83ab-47fe-bb87-03d1dfc90fa5 None None] MFA Rules not processed for user `318f2d1fb49e4b2dac41b61b1108f32b`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:58.082323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f1943304-83ab-47fe-bb87-03d1dfc90fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:58.107693 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601a892-7d62-41d7-b880-8e7bea8395fd admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:58.111852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8886fa2-7dad-4435-8b64-774f5ce27594 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:58.112444 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8886fa2-7dad-4435-8b64-774f5ce27594 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:58.112769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8886fa2-7dad-4435-8b64-774f5ce27594 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:58.113044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8886fa2-7dad-4435-8b64-774f5ce27594 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:58.119172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:58.249500 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 892/1806] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:58 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:58.250617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1943304-83ab-47fe-bb87-03d1dfc90fa5 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:58.284648 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8886fa2-7dad-4435-8b64-774f5ce27594 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:58.288145 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8886fa2-7dad-4435-8b64-774f5ce27594 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:58.305585 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 915/1807] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:58.307020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8886fa2-7dad-4435-8b64-774f5ce27594 service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:58.308413 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c8886fa2-7dad-4435-8b64-774f5ce27594 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:58.320384 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8886fa2-7dad-4435-8b64-774f5ce27594 service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:58.327502 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e8ee2-9685-4e90-84bc-8047498066c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:58.328646 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e8ee2-9685-4e90-84bc-8047498066c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:58.329798 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e8ee2-9685-4e90-84bc-8047498066c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:58.330080 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e8ee2-9685-4e90-84bc-8047498066c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:58.344901 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e8ee2-9685-4e90-84bc-8047498066c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:58.346730 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e8ee2-9685-4e90-84bc-8047498066c3 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:58.361304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 916/1808] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:58 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:23:58.369646 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1943304-83ab-47fe-bb87-03d1dfc90fa5 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:58.394024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a3f302-512f-4389-9463-fcb9223f7758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:58.395040 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3f302-512f-4389-9463-fcb9223f7758 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:58.395478 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3f302-512f-4389-9463-fcb9223f7758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:58.395917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3f302-512f-4389-9463-fcb9223f7758 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:58.396794 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a3f302-512f-4389-9463-fcb9223f7758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:58.399059 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a3f302-512f-4389-9463-fcb9223f7758 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:58.460756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 893/1809] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:58 2022] POST /identity/v3/projects => generated 385 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:58.465255 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e8ee2-9685-4e90-84bc-8047498066c3 service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:58.513335 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e8ee2-9685-4e90-84bc-8047498066c3 service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:58.727581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97caa320-e96d-4e7b-bbe4-ae727f339711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:58.729968 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97caa320-e96d-4e7b-bbe4-ae727f339711 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:58.730257 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97caa320-e96d-4e7b-bbe4-ae727f339711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:58.731931 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97caa320-e96d-4e7b-bbe4-ae727f339711 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:58.734111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97caa320-e96d-4e7b-bbe4-ae727f339711 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:58.739200 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97caa320-e96d-4e7b-bbe4-ae727f339711 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:58.793102 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 917/1810] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:58 2022] POST /identity/v3/users => generated 293 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:58.794518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a3f302-512f-4389-9463-fcb9223f7758 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:58.817561 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a3f302-512f-4389-9463-fcb9223f7758 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:58.832523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b6624-1693-4fbf-81fb-60177de85739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:58.841060 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b6624-1693-4fbf-81fb-60177de85739 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:58.849231 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b6624-1693-4fbf-81fb-60177de85739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:58.851041 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b6624-1693-4fbf-81fb-60177de85739 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:58.854738 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923b6624-1693-4fbf-81fb-60177de85739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:58.856643 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923b6624-1693-4fbf-81fb-60177de85739 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:58.888864 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 894/1811] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:58.889731 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97caa320-e96d-4e7b-bbe4-ae727f339711 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:58.892927 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b6624-1693-4fbf-81fb-60177de85739 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:58.912641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b6624-1693-4fbf-81fb-60177de85739 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:58.918342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d085f-56e2-4d3c-876f-b253b176e04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:58.919415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d085f-56e2-4d3c-876f-b253b176e04e admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:58.919939 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d085f-56e2-4d3c-876f-b253b176e04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:58.920428 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d085f-56e2-4d3c-876f-b253b176e04e admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:58.921316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d085f-56e2-4d3c-876f-b253b176e04e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:58.923465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d085f-56e2-4d3c-876f-b253b176e04e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:58.947732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 895/1812] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:23:58.958544 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d085f-56e2-4d3c-876f-b253b176e04e admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:58.985669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d085f-56e2-4d3c-876f-b253b176e04e admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:59.001261 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe9f4d1-afde-42b9-8335-44a25120ba20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:59.003391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9f4d1-afde-42b9-8335-44a25120ba20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:59.003644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9f4d1-afde-42b9-8335-44a25120ba20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:59.004149 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9f4d1-afde-42b9-8335-44a25120ba20 admin admin] PATH_INFO: `/v3/projects/5af56ecc01d441aba9921b569cf0443e/users/60d6882240324539a721cbfd82f04469/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:59.039713 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe9f4d1-afde-42b9-8335-44a25120ba20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5af56ecc01d441aba9921b569cf0443e, user_id=60d6882240324539a721cbfd82f04469, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:59.045981 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe9f4d1-afde-42b9-8335-44a25120ba20 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:59.051981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97caa320-e96d-4e7b-bbe4-ae727f339711 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:59.057559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471b33b-4012-4a20-8831-a6569d9420b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:59.058188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471b33b-4012-4a20-8831-a6569d9420b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:59.058470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471b33b-4012-4a20-8831-a6569d9420b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:59.058744 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471b33b-4012-4a20-8831-a6569d9420b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:59.061863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3471b33b-4012-4a20-8831-a6569d9420b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:59.077150 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 896/1813] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:23:58 2022] PUT /identity/v3/projects/5af56ecc01d441aba9921b569cf0443e/users/60d6882240324539a721cbfd82f04469/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:59.089873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2325575-647f-4303-b87a-2e00df2102d7 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:59.090282 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2325575-647f-4303-b87a-2e00df2102d7 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:59.090665 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2325575-647f-4303-b87a-2e00df2102d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:59.121942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:59.215479 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:59.253238 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a2325575-647f-4303-b87a-2e00df2102d7 None None] MFA Rules not processed for user `60d6882240324539a721cbfd82f04469`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:59.259627 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a2325575-647f-4303-b87a-2e00df2102d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:59.336896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471b33b-4012-4a20-8831-a6569d9420b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:59.339576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471b33b-4012-4a20-8831-a6569d9420b5 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:59.370621 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 918/1814] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:58 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:23:59.383841 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471b33b-4012-4a20-8831-a6569d9420b5 service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:59.394250 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3471b33b-4012-4a20-8831-a6569d9420b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:59.433505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471b33b-4012-4a20-8831-a6569d9420b5 service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:59.447921 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 897/1815] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:59.452233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2325575-647f-4303-b87a-2e00df2102d7 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:59.510090 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee254a-d6ba-4e4a-ab1c-3b1bf0d34367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:59.511136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee254a-d6ba-4e4a-ab1c-3b1bf0d34367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:59.511578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee254a-d6ba-4e4a-ab1c-3b1bf0d34367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:59.512267 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee254a-d6ba-4e4a-ab1c-3b1bf0d34367 admin admin] PATH_INFO: `/v3/users/833bfed6f72b419281678aca7c28b2e4` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:59.513722 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee254a-d6ba-4e4a-ab1c-3b1bf0d34367 admin admin] RBAC: Authorizing `identity:delete_user(user_id=833bfed6f72b419281678aca7c28b2e4)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:59.518372 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee254a-d6ba-4e4a-ab1c-3b1bf0d34367 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:59.545462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2325575-647f-4303-b87a-2e00df2102d7 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:59.553438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939410d-529a-47de-a50f-f71c6bd0e2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:59.554250 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939410d-529a-47de-a50f-f71c6bd0e2b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:59.554657 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939410d-529a-47de-a50f-f71c6bd0e2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:59.555152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939410d-529a-47de-a50f-f71c6bd0e2b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:59.556678 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9939410d-529a-47de-a50f-f71c6bd0e2b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:59.559744 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9939410d-529a-47de-a50f-f71c6bd0e2b8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:59.631874 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-03ee254a-d6ba-4e4a-ab1c-3b1bf0d34367 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '833bfed6f72b419281678aca7c28b2e4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:59.632587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-03ee254a-d6ba-4e4a-ab1c-3b1bf0d34367 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '833bfed6f72b419281678aca7c28b2e4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:59.633232 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-03ee254a-d6ba-4e4a-ab1c-3b1bf0d34367 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '833bfed6f72b419281678aca7c28b2e4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:59.637859 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 898/1816] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:23:59 2022] POST /identity/v3/projects => generated 387 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:59.644919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939410d-529a-47de-a50f-f71c6bd0e2b8 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:59.648870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9939410d-529a-47de-a50f-f71c6bd0e2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:59.655540 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-03ee254a-d6ba-4e4a-ab1c-3b1bf0d34367 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '833bfed6f72b419281678aca7c28b2e4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:59.677569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-03ee254a-d6ba-4e4a-ab1c-3b1bf0d34367 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '833bfed6f72b419281678aca7c28b2e4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:23:59.705565 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 919/1817] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:23:59 2022] DELETE /identity/v3/users/833bfed6f72b419281678aca7c28b2e4 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:23:59.708187 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c50b7-0428-4534-929e-d73c2f160ff5 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:59.708527 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c50b7-0428-4534-929e-d73c2f160ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:59.709064 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c50b7-0428-4534-929e-d73c2f160ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:59.741168 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-504c50b7-0428-4534-929e-d73c2f160ff5 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:23:59.742551 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-504c50b7-0428-4534-929e-d73c2f160ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:59.809016 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939410d-529a-47de-a50f-f71c6bd0e2b8 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:23:59.847522 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:59.894029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:23:59.929754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e33010-9c20-4525-b1da-d856a5d7089a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:23:59.932139 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e33010-9c20-4525-b1da-d856a5d7089a admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:23:59.932430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e33010-9c20-4525-b1da-d856a5d7089a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:23:59.932707 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e33010-9c20-4525-b1da-d856a5d7089a admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:23:59.933359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e33010-9c20-4525-b1da-d856a5d7089a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:23:59.935830 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 920/1818] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:23:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:23:59.941169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e33010-9c20-4525-b1da-d856a5d7089a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:23:59.981701 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c50b7-0428-4534-929e-d73c2f160ff5 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:23:59.983387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-504c50b7-0428-4534-929e-d73c2f160ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:23:59.986825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 899/1819] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:23:59 2022] POST /identity/v3/users => generated 300 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:23:59.991598 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e33010-9c20-4525-b1da-d856a5d7089a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:00.018482 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e33010-9c20-4525-b1da-d856a5d7089a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:00.036723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9688f5-fe39-46c7-9826-928a1b99cf52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:00.037974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9688f5-fe39-46c7-9826-928a1b99cf52 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:00.038283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9688f5-fe39-46c7-9826-928a1b99cf52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:00.038598 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9688f5-fe39-46c7-9826-928a1b99cf52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:00.039274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9688f5-fe39-46c7-9826-928a1b99cf52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:00.041731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9688f5-fe39-46c7-9826-928a1b99cf52 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:00.071974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 900/1820] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:00.073710 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1d476-3944-408b-ac40-ee013f195b80 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:00.074003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1d476-3944-408b-ac40-ee013f195b80 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:00.074281 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1d476-3944-408b-ac40-ee013f195b80 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:00.082876 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 901/1821] 10.222.0.81 () {54 vars in 921 bytes} [Mon Sep 19 11:24:00 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:24:00.084325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1d476-3944-408b-ac40-ee013f195b80 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:00.126874 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1d476-3944-408b-ac40-ee013f195b80 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:00.162693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dee3d5-42bf-4818-a2a7-ce5cb4ccdaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:00.163447 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dee3d5-42bf-4818-a2a7-ce5cb4ccdaae admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:00.163745 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dee3d5-42bf-4818-a2a7-ce5cb4ccdaae admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:00.164049 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dee3d5-42bf-4818-a2a7-ce5cb4ccdaae admin admin] PATH_INFO: `/v3/projects/c8a65efaac354355bc6659826136f3cb/users/603e936d370f4c5d98f0216f2947e576/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:00.201225 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c50b7-0428-4534-929e-d73c2f160ff5 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:00.217299 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832389d-cb4b-4b49-a375-2ad43cc87c56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:00.218546 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832389d-cb4b-4b49-a375-2ad43cc87c56 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:00.220976 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832389d-cb4b-4b49-a375-2ad43cc87c56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:00.220976 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832389d-cb4b-4b49-a375-2ad43cc87c56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:00.229324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dee3d5-42bf-4818-a2a7-ce5cb4ccdaae admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8a65efaac354355bc6659826136f3cb, user_id=603e936d370f4c5d98f0216f2947e576, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:00.238318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dee3d5-42bf-4818-a2a7-ce5cb4ccdaae admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:00.245828 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e832389d-cb4b-4b49-a375-2ad43cc87c56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:00.248633 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e832389d-cb4b-4b49-a375-2ad43cc87c56 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:00.274014 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 902/1822] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:00 2022] PUT /identity/v3/projects/c8a65efaac354355bc6659826136f3cb/users/603e936d370f4c5d98f0216f2947e576/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:00.277463 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dee3d5-42bf-4818-a2a7-ce5cb4ccdaae admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:00.278921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 921/1823] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:23:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:00.321519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb81a47-2448-41df-8456-6680db9ea731 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:00.322351 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb81a47-2448-41df-8456-6680db9ea731 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:00.322587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb81a47-2448-41df-8456-6680db9ea731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:00.373568 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-fdb81a47-2448-41df-8456-6680db9ea731 None None] MFA Rules not processed for user `5abd70a18ccd407bafe3b61c8870986e`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:00.377007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-fdb81a47-2448-41df-8456-6680db9ea731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:00.462615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dee3d5-42bf-4818-a2a7-ce5cb4ccdaae admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:00.494077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8194b2-c3bc-4c21-b0d4-05561a4e5a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:00.494408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8194b2-c3bc-4c21-b0d4-05561a4e5a91 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:00.494934 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8194b2-c3bc-4c21-b0d4-05561a4e5a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:00.495210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8194b2-c3bc-4c21-b0d4-05561a4e5a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:00.496395 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8194b2-c3bc-4c21-b0d4-05561a4e5a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:00.498738 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8194b2-c3bc-4c21-b0d4-05561a4e5a91 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:00.532630 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 903/1824] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:00.532630 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8194b2-c3bc-4c21-b0d4-05561a4e5a91 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:00.544729 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ab8194b2-c3bc-4c21-b0d4-05561a4e5a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:00.578719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8194b2-c3bc-4c21-b0d4-05561a4e5a91 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:00.586291 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d62993-bbd6-43bc-800f-cb5c7d03ded8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:00.588323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d62993-bbd6-43bc-800f-cb5c7d03ded8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:00.589263 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d62993-bbd6-43bc-800f-cb5c7d03ded8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:00.589722 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d62993-bbd6-43bc-800f-cb5c7d03ded8 admin admin] PATH_INFO: `/v3/projects/dc45dc6bc8fd4939b8e7af39e1f40f8f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:00.591169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d62993-bbd6-43bc-800f-cb5c7d03ded8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc45dc6bc8fd4939b8e7af39e1f40f8f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:00.593552 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d62993-bbd6-43bc-800f-cb5c7d03ded8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:00.609762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 922/1825] 10.222.0.81 () {58 vars in 1018 bytes} [Mon Sep 19 11:24:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:24:00.633553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb81a47-2448-41df-8456-6680db9ea731 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:00.651409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb81a47-2448-41df-8456-6680db9ea731 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:00.661712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388a32a-5219-475c-9b53-476023ded44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:00.663128 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388a32a-5219-475c-9b53-476023ded44a admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:00.663418 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388a32a-5219-475c-9b53-476023ded44a admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:00.664071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388a32a-5219-475c-9b53-476023ded44a admin admin] PATH_INFO: `/v3/projects/c8a65efaac354355bc6659826136f3cb/users/603e936d370f4c5d98f0216f2947e576/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:00.670698 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388a32a-5219-475c-9b53-476023ded44a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8a65efaac354355bc6659826136f3cb, user_id=603e936d370f4c5d98f0216f2947e576, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:00.675902 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388a32a-5219-475c-9b53-476023ded44a admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:00.698330 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c5d62993-bbd6-43bc-800f-cb5c7d03ded8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc45dc6bc8fd4939b8e7af39e1f40f8f', 'request_id': 'req-c5d62993-bbd6-43bc-800f-cb5c7d03ded8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:00.708286 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c5d62993-bbd6-43bc-800f-cb5c7d03ded8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc45dc6bc8fd4939b8e7af39e1f40f8f', 'request_id': 'req-c5d62993-bbd6-43bc-800f-cb5c7d03ded8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:00.708525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 923/1826] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:00 2022] PUT /identity/v3/projects/c8a65efaac354355bc6659826136f3cb/users/603e936d370f4c5d98f0216f2947e576/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:00.709798 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388a32a-5219-475c-9b53-476023ded44a admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:00.717190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c5d62993-bbd6-43bc-800f-cb5c7d03ded8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc45dc6bc8fd4939b8e7af39e1f40f8f', 'request_id': 'req-c5d62993-bbd6-43bc-800f-cb5c7d03ded8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:00.725345 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c388a32a-5219-475c-9b53-476023ded44a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:00.737966 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:00.748362 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c5d62993-bbd6-43bc-800f-cb5c7d03ded8 admin admin] The token cache is being invalidate because project dc45dc6bc8fd4939b8e7af39e1f40f8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:00.749452 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c5d62993-bbd6-43bc-800f-cb5c7d03ded8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:00.752190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c5d62993-bbd6-43bc-800f-cb5c7d03ded8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:00.761937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 904/1827] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:00 2022] DELETE /identity/v3/projects/dc45dc6bc8fd4939b8e7af39e1f40f8f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:00.762317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0aee7d-4aab-43e4-b276-1ddad7f3505a None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:00.763352 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0aee7d-4aab-43e4-b276-1ddad7f3505a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:00.765199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0aee7d-4aab-43e4-b276-1ddad7f3505a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:00.841251 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-4b0aee7d-4aab-43e4-b276-1ddad7f3505a None None] MFA Rules not processed for user `603e936d370f4c5d98f0216f2947e576`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:00.843271 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4b0aee7d-4aab-43e4-b276-1ddad7f3505a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:00.866154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388a32a-5219-475c-9b53-476023ded44a admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:00.875667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8429421-d655-4ce4-bfd2-de6793bcbf97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:00.878821 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8429421-d655-4ce4-bfd2-de6793bcbf97 service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:00.879224 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8429421-d655-4ce4-bfd2-de6793bcbf97 service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:00.879539 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8429421-d655-4ce4-bfd2-de6793bcbf97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:00.883932 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e8429421-d655-4ce4-bfd2-de6793bcbf97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:00.916862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8429421-d655-4ce4-bfd2-de6793bcbf97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:00.918511 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8429421-d655-4ce4-bfd2-de6793bcbf97 service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:00.934673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:00.961432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 924/1828] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:00.969473 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c875c23-3b7c-42fc-b600-6e56f5664030 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:00.969867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c875c23-3b7c-42fc-b600-6e56f5664030 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:00.969973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c875c23-3b7c-42fc-b600-6e56f5664030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:00.982685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 905/1829] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:00.994502 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0aee7d-4aab-43e4-b276-1ddad7f3505a None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:00.996840 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4b0aee7d-4aab-43e4-b276-1ddad7f3505a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:01.029791 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-2c875c23-3b7c-42fc-b600-6e56f5664030 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:01.031038 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2c875c23-3b7c-42fc-b600-6e56f5664030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:01.072986 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0aee7d-4aab-43e4-b276-1ddad7f3505a None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:01.077085 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41484da5-9bd4-4286-bf50-5f6f51f98906 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:01.077716 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41484da5-9bd4-4286-bf50-5f6f51f98906 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:01.078169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41484da5-9bd4-4286-bf50-5f6f51f98906 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:01.078659 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41484da5-9bd4-4286-bf50-5f6f51f98906 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:01.092054 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41484da5-9bd4-4286-bf50-5f6f51f98906 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:01.094160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41484da5-9bd4-4286-bf50-5f6f51f98906 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:01.107597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 906/1830] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:01.123012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa31c9b-7204-45ed-822d-4e6432d95f1a None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:01.123362 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa31c9b-7204-45ed-822d-4e6432d95f1a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:01.123922 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa31c9b-7204-45ed-822d-4e6432d95f1a None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:01.125694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 907/1831] 10.222.0.81 () {54 vars in 921 bytes} [Mon Sep 19 11:24:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:24:01.153081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d6a55-cf72-4cad-b340-063f57136a25 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:01.153373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d6a55-cf72-4cad-b340-063f57136a25 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:01.153652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d6a55-cf72-4cad-b340-063f57136a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:01.212309 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-602d6a55-cf72-4cad-b340-063f57136a25 None None] MFA Rules not processed for user `5abd70a18ccd407bafe3b61c8870986e`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:01.215067 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-602d6a55-cf72-4cad-b340-063f57136a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:01.250418 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 908/1832] 10.222.0.81 () {58 vars in 1018 bytes} [Mon Sep 19 11:24:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:01.253725 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d6a55-cf72-4cad-b340-063f57136a25 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:01.259610 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-602d6a55-cf72-4cad-b340-063f57136a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:01.288443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 925/1833] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:01.305802 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07643d-0f6a-4a27-9fca-dc7667bed1fe None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:01.307183 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07643d-0f6a-4a27-9fca-dc7667bed1fe None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:01.308655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07643d-0f6a-4a27-9fca-dc7667bed1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:01.357059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-ed07643d-0f6a-4a27-9fca-dc7667bed1fe None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:01.362724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ed07643d-0f6a-4a27-9fca-dc7667bed1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:01.386561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 926/1834] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:01.391318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07643d-0f6a-4a27-9fca-dc7667bed1fe None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:01.424967 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07643d-0f6a-4a27-9fca-dc7667bed1fe None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:01.430514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1777a26-8922-4624-a355-cd0bb8abdaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:01.431343 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1777a26-8922-4624-a355-cd0bb8abdaf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:01.431883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1777a26-8922-4624-a355-cd0bb8abdaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:01.432225 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1777a26-8922-4624-a355-cd0bb8abdaf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:01.432976 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1777a26-8922-4624-a355-cd0bb8abdaf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:01.433441 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1777a26-8922-4624-a355-cd0bb8abdaf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:01.435420 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1777a26-8922-4624-a355-cd0bb8abdaf9 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:01.445096 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d6a55-cf72-4cad-b340-063f57136a25 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:01.454703 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 927/1835] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:01.457251 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36475f6-b1c8-4f9f-a5ed-b63049e1fa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:01.458564 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36475f6-b1c8-4f9f-a5ed-b63049e1fa7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:01.459263 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36475f6-b1c8-4f9f-a5ed-b63049e1fa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:01.459549 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36475f6-b1c8-4f9f-a5ed-b63049e1fa7c admin admin] PATH_INFO: `/v3/users/60d6882240324539a721cbfd82f04469` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:01.460011 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1777a26-8922-4624-a355-cd0bb8abdaf9 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:01.461057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36475f6-b1c8-4f9f-a5ed-b63049e1fa7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=60d6882240324539a721cbfd82f04469)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:01.464007 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36475f6-b1c8-4f9f-a5ed-b63049e1fa7c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:01.489411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1777a26-8922-4624-a355-cd0bb8abdaf9 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:01.502708 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326ce40-1f59-45af-944a-095ed115e5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:01.510154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326ce40-1f59-45af-944a-095ed115e5af admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:01.510492 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326ce40-1f59-45af-944a-095ed115e5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:01.510770 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326ce40-1f59-45af-944a-095ed115e5af admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:01.512484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326ce40-1f59-45af-944a-095ed115e5af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:01.514925 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326ce40-1f59-45af-944a-095ed115e5af admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:01.548857 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:01.594311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 928/1836] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:01 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:01.595355 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326ce40-1f59-45af-944a-095ed115e5af admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:01.614284 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b36475f6-b1c8-4f9f-a5ed-b63049e1fa7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60d6882240324539a721cbfd82f04469'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:01.633182 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b36475f6-b1c8-4f9f-a5ed-b63049e1fa7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60d6882240324539a721cbfd82f04469'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:01.634229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b36475f6-b1c8-4f9f-a5ed-b63049e1fa7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60d6882240324539a721cbfd82f04469'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:01.635278 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b36475f6-b1c8-4f9f-a5ed-b63049e1fa7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60d6882240324539a721cbfd82f04469'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:01.667370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b36475f6-b1c8-4f9f-a5ed-b63049e1fa7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60d6882240324539a721cbfd82f04469'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:01.697721 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 909/1837] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:01 2022] DELETE /identity/v3/users/60d6882240324539a721cbfd82f04469 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:01.702253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d614945-2089-4bf9-8893-0cb33a70519e None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:01.702840 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d614945-2089-4bf9-8893-0cb33a70519e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:01.703130 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d614945-2089-4bf9-8893-0cb33a70519e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:01.764951 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-4d614945-2089-4bf9-8893-0cb33a70519e None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:01.766252 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4d614945-2089-4bf9-8893-0cb33a70519e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:01.776416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326ce40-1f59-45af-944a-095ed115e5af admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:01.861433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 910/1838] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:24:01.877473 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138d4c0-ef2b-4078-b114-55a4619ee1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:01.878094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138d4c0-ef2b-4078-b114-55a4619ee1af admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:01.878377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138d4c0-ef2b-4078-b114-55a4619ee1af admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:01.878651 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138d4c0-ef2b-4078-b114-55a4619ee1af admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:01.879840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138d4c0-ef2b-4078-b114-55a4619ee1af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:01.881579 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d614945-2089-4bf9-8893-0cb33a70519e None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:01.881889 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138d4c0-ef2b-4078-b114-55a4619ee1af admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:01.883303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4d614945-2089-4bf9-8893-0cb33a70519e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:01.898782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:01.928216 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 929/1839] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:01 2022] POST /identity/v3/users => generated 294 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:01.928427 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138d4c0-ef2b-4078-b114-55a4619ee1af admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:01.931589 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a138d4c0-ef2b-4078-b114-55a4619ee1af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:02.034293 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d614945-2089-4bf9-8893-0cb33a70519e None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:02.048945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138d4c0-ef2b-4078-b114-55a4619ee1af admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:02.051070 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d7966-7c18-41eb-b105-1754e4b9f254 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:02.052714 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d7966-7c18-41eb-b105-1754e4b9f254 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:02.053113 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d7966-7c18-41eb-b105-1754e4b9f254 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:02.053497 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d7966-7c18-41eb-b105-1754e4b9f254 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:02.054956 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddfe760-8f4d-4c89-9f58-7dd953ec0c2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:02.055582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfe760-8f4d-4c89-9f58-7dd953ec0c2c service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:02.055754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-372d7966-7c18-41eb-b105-1754e4b9f254 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:02.055975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfe760-8f4d-4c89-9f58-7dd953ec0c2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:02.056403 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfe760-8f4d-4c89-9f58-7dd953ec0c2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:02.068837 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddfe760-8f4d-4c89-9f58-7dd953ec0c2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:02.070958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddfe760-8f4d-4c89-9f58-7dd953ec0c2c service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:02.076641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d7966-7c18-41eb-b105-1754e4b9f254 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:02.078254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d7966-7c18-41eb-b105-1754e4b9f254 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:02.082916 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 930/1840] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:02.084306 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddfe760-8f4d-4c89-9f58-7dd953ec0c2c service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:02.086901 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2ddfe760-8f4d-4c89-9f58-7dd953ec0c2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:02.090901 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 911/1841] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:02.101422 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddfe760-8f4d-4c89-9f58-7dd953ec0c2c service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:02.107887 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c642671-8edf-4585-897e-2d266a7a6a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:02.108500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c642671-8edf-4585-897e-2d266a7a6a40 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:02.108782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c642671-8edf-4585-897e-2d266a7a6a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:02.109112 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c642671-8edf-4585-897e-2d266a7a6a40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:02.109971 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c642671-8edf-4585-897e-2d266a7a6a40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:02.111574 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c642671-8edf-4585-897e-2d266a7a6a40 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:02.130293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 931/1842] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:02.132305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d7966-7c18-41eb-b105-1754e4b9f254 service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:02.145267 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d7966-7c18-41eb-b105-1754e4b9f254 service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:02.151202 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d2fca-2f2e-4037-8987-fbe46a962fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:02.151644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d2fca-2f2e-4037-8987-fbe46a962fcc admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:02.152022 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d2fca-2f2e-4037-8987-fbe46a962fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:02.152392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d2fca-2f2e-4037-8987-fbe46a962fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:02.153858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d2fca-2f2e-4037-8987-fbe46a962fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:02.156003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d2fca-2f2e-4037-8987-fbe46a962fcc admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:02.174745 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c642671-8edf-4585-897e-2d266a7a6a40 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:02.176022 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 912/1843] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:02.187340 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d2fca-2f2e-4037-8987-fbe46a962fcc admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:02.191150 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c642671-8edf-4585-897e-2d266a7a6a40 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:02.200474 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-884d2fca-2f2e-4037-8987-fbe46a962fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:02.210991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58a612b-5f10-4288-95bb-789bc732667a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:02.211583 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a612b-5f10-4288-95bb-789bc732667a admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:02.211896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a612b-5f10-4288-95bb-789bc732667a admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:02.212757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a612b-5f10-4288-95bb-789bc732667a admin admin] PATH_INFO: `/v3/projects/14d8e11e839c4d88b7612b826619ecc6/users/3369f0db545d449cbfb8d187efab5675/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:02.235222 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d2fca-2f2e-4037-8987-fbe46a962fcc admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:02.246571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58a612b-5f10-4288-95bb-789bc732667a admin admin] RBAC: Authorizing `identity:create_grant(project_id=14d8e11e839c4d88b7612b826619ecc6, user_id=3369f0db545d449cbfb8d187efab5675, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:02.249892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749a1b8-c783-4fef-bfd0-bcfdc06a55be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:02.254480 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749a1b8-c783-4fef-bfd0-bcfdc06a55be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:02.257740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749a1b8-c783-4fef-bfd0-bcfdc06a55be admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:02.258439 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749a1b8-c783-4fef-bfd0-bcfdc06a55be admin admin] PATH_INFO: `/v3/projects/cdf9ae08ece14e2baf56fdf8b4ea2c39` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:02.263069 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58a612b-5f10-4288-95bb-789bc732667a admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:02.265662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749a1b8-c783-4fef-bfd0-bcfdc06a55be admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdf9ae08ece14e2baf56fdf8b4ea2c39)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:02.270493 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749a1b8-c783-4fef-bfd0-bcfdc06a55be admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:02.294717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 932/1844] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:02 2022] PUT /identity/v3/projects/14d8e11e839c4d88b7612b826619ecc6/users/3369f0db545d449cbfb8d187efab5675/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:02.305642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a598e-b3dd-431e-9559-988271ca4371 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:02.306474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a598e-b3dd-431e-9559-988271ca4371 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:02.307838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a598e-b3dd-431e-9559-988271ca4371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:02.318048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:02.378411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4749a1b8-c783-4fef-bfd0-bcfdc06a55be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdf9ae08ece14e2baf56fdf8b4ea2c39', 'request_id': 'req-4749a1b8-c783-4fef-bfd0-bcfdc06a55be'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:02.388289 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4749a1b8-c783-4fef-bfd0-bcfdc06a55be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdf9ae08ece14e2baf56fdf8b4ea2c39', 'request_id': 'req-4749a1b8-c783-4fef-bfd0-bcfdc06a55be'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:02.395364 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-7a7a598e-b3dd-431e-9559-988271ca4371 None None] MFA Rules not processed for user `3369f0db545d449cbfb8d187efab5675`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:02.397469 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-7a7a598e-b3dd-431e-9559-988271ca4371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:02.401147 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4749a1b8-c783-4fef-bfd0-bcfdc06a55be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdf9ae08ece14e2baf56fdf8b4ea2c39', 'request_id': 'req-4749a1b8-c783-4fef-bfd0-bcfdc06a55be'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:02.424515 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4749a1b8-c783-4fef-bfd0-bcfdc06a55be admin admin] The token cache is being invalidate because project cdf9ae08ece14e2baf56fdf8b4ea2c39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:02.425246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4749a1b8-c783-4fef-bfd0-bcfdc06a55be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:02.426414 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4749a1b8-c783-4fef-bfd0-bcfdc06a55be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:02.433070 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 913/1845] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:02 2022] DELETE /identity/v3/projects/cdf9ae08ece14e2baf56fdf8b4ea2c39 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:02.435196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749a1b8-c783-4fef-bfd0-bcfdc06a55be admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:02.439793 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4749a1b8-c783-4fef-bfd0-bcfdc06a55be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:02.515241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 933/1846] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:24:02.520447 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a598e-b3dd-431e-9559-988271ca4371 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:02.522215 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-7a7a598e-b3dd-431e-9559-988271ca4371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:02.543934 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749a1b8-c783-4fef-bfd0-bcfdc06a55be admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:02.549947 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b71ea-a6d8-44e9-acfd-0510ac6d8f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:02.550590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b71ea-a6d8-44e9-acfd-0510ac6d8f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:02.550872 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b71ea-a6d8-44e9-acfd-0510ac6d8f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:02.551146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b71ea-a6d8-44e9-acfd-0510ac6d8f9d admin admin] PATH_INFO: `/v3/users/0fcec5c32aaf4d679db48a496847eaf8` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:02.552771 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b71ea-a6d8-44e9-acfd-0510ac6d8f9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fcec5c32aaf4d679db48a496847eaf8)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:02.556772 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b71ea-a6d8-44e9-acfd-0510ac6d8f9d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:02.586201 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:02.610934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a598e-b3dd-431e-9559-988271ca4371 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:02.616992 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdccddb6-0702-44d7-b642-f715cdbbd0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:02.617707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccddb6-0702-44d7-b642-f715cdbbd0ef admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:02.618817 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccddb6-0702-44d7-b642-f715cdbbd0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:02.619197 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccddb6-0702-44d7-b642-f715cdbbd0ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:02.620891 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdccddb6-0702-44d7-b642-f715cdbbd0ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:02.623039 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdccddb6-0702-44d7-b642-f715cdbbd0ef admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:02.623213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-de4b71ea-a6d8-44e9-acfd-0510ac6d8f9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fcec5c32aaf4d679db48a496847eaf8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:02.637755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-de4b71ea-a6d8-44e9-acfd-0510ac6d8f9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fcec5c32aaf4d679db48a496847eaf8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:02.638369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-de4b71ea-a6d8-44e9-acfd-0510ac6d8f9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fcec5c32aaf4d679db48a496847eaf8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:02.639486 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-de4b71ea-a6d8-44e9-acfd-0510ac6d8f9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fcec5c32aaf4d679db48a496847eaf8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:02.663870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-de4b71ea-a6d8-44e9-acfd-0510ac6d8f9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fcec5c32aaf4d679db48a496847eaf8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:02.680656 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 934/1847] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:02 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:24:02.682103 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdccddb6-0702-44d7-b642-f715cdbbd0ef admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:02.686168 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-fdccddb6-0702-44d7-b642-f715cdbbd0ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:02.687256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 914/1848] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:02 2022] DELETE /identity/v3/users/0fcec5c32aaf4d679db48a496847eaf8 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:02.688201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b71ea-a6d8-44e9-acfd-0510ac6d8f9d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:02.693347 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-de4b71ea-a6d8-44e9-acfd-0510ac6d8f9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:02.733948 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:02.792225 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdccddb6-0702-44d7-b642-f715cdbbd0ef admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:02.832876 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b71ea-a6d8-44e9-acfd-0510ac6d8f9d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:02.845822 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa201107-c656-4784-94c5-b99f8c015803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:02.847754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa201107-c656-4784-94c5-b99f8c015803 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:02.848071 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa201107-c656-4784-94c5-b99f8c015803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:02.848353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa201107-c656-4784-94c5-b99f8c015803 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:02.849017 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa201107-c656-4784-94c5-b99f8c015803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:02.851302 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa201107-c656-4784-94c5-b99f8c015803 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:02.884825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521927c-2b99-4c6c-89bd-ce1e104181d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:02.885952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521927c-2b99-4c6c-89bd-ce1e104181d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:02.886403 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521927c-2b99-4c6c-89bd-ce1e104181d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:02.886977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521927c-2b99-4c6c-89bd-ce1e104181d7 admin admin] PATH_INFO: `/v3/projects/5af56ecc01d441aba9921b569cf0443e` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:02.888806 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a521927c-2b99-4c6c-89bd-ce1e104181d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5af56ecc01d441aba9921b569cf0443e)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:02.891794 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a521927c-2b99-4c6c-89bd-ce1e104181d7 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:02.910102 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 915/1849] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:02 2022] POST /identity/v3/users => generated 300 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:02.916822 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa201107-c656-4784-94c5-b99f8c015803 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:02.941138 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa201107-c656-4784-94c5-b99f8c015803 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:02.959346 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a521927c-2b99-4c6c-89bd-ce1e104181d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5af56ecc01d441aba9921b569cf0443e', 'request_id': 'req-a521927c-2b99-4c6c-89bd-ce1e104181d7'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:02.973948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a521927c-2b99-4c6c-89bd-ce1e104181d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5af56ecc01d441aba9921b569cf0443e', 'request_id': 'req-a521927c-2b99-4c6c-89bd-ce1e104181d7'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:02.990067 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a521927c-2b99-4c6c-89bd-ce1e104181d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5af56ecc01d441aba9921b569cf0443e', 'request_id': 'req-a521927c-2b99-4c6c-89bd-ce1e104181d7'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:03.021383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a521927c-2b99-4c6c-89bd-ce1e104181d7 admin admin] The token cache is being invalidate because project 5af56ecc01d441aba9921b569cf0443e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:03.021840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a521927c-2b99-4c6c-89bd-ce1e104181d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:03.025468 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a521927c-2b99-4c6c-89bd-ce1e104181d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:03.027094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 935/1850] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:02 2022] DELETE /identity/v3/projects/5af56ecc01d441aba9921b569cf0443e => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:03.031269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521927c-2b99-4c6c-89bd-ce1e104181d7 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:03.033136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a521927c-2b99-4c6c-89bd-ce1e104181d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:03.049421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb6f22-79c6-499c-a977-33056aeb469e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:03.050144 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb6f22-79c6-499c-a977-33056aeb469e admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:03.050916 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb6f22-79c6-499c-a977-33056aeb469e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:03.050916 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb6f22-79c6-499c-a977-33056aeb469e admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:03.051691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb6f22-79c6-499c-a977-33056aeb469e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:03.053977 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb6f22-79c6-499c-a977-33056aeb469e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:03.058947 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:03.068650 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 916/1851] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:03.072593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb6f22-79c6-499c-a977-33056aeb469e admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:03.075068 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-febb6f22-79c6-499c-a977-33056aeb469e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:03.093628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb6f22-79c6-499c-a977-33056aeb469e admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:03.102296 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89795a-6536-4aae-8c14-386066b38d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:03.104307 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89795a-6536-4aae-8c14-386066b38d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:03.105028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89795a-6536-4aae-8c14-386066b38d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:03.105644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89795a-6536-4aae-8c14-386066b38d7d admin admin] PATH_INFO: `/v3/projects/b2a480dfbca74a1c8b15df44baf56665/users/4eeee299e8c0430493f53bc98c0037fa/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:03.127548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89795a-6536-4aae-8c14-386066b38d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a480dfbca74a1c8b15df44baf56665, user_id=4eeee299e8c0430493f53bc98c0037fa, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:03.130935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89795a-6536-4aae-8c14-386066b38d7d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:03.131530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521927c-2b99-4c6c-89bd-ce1e104181d7 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:03.138198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b9c837-f1b9-4373-abcc-5c013cf834f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:03.138819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9c837-f1b9-4373-abcc-5c013cf834f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:03.139204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9c837-f1b9-4373-abcc-5c013cf834f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:03.139576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9c837-f1b9-4373-abcc-5c013cf834f5 admin admin] PATH_INFO: `/v3/users/603e936d370f4c5d98f0216f2947e576` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:03.141030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b9c837-f1b9-4373-abcc-5c013cf834f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=603e936d370f4c5d98f0216f2947e576)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:03.143314 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b9c837-f1b9-4373-abcc-5c013cf834f5 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:03.154978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 917/1852] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:03 2022] PUT /identity/v3/projects/b2a480dfbca74a1c8b15df44baf56665/users/4eeee299e8c0430493f53bc98c0037fa/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:03.159562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89795a-6536-4aae-8c14-386066b38d7d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:03.237729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-d3b9c837-f1b9-4373-abcc-5c013cf834f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '603e936d370f4c5d98f0216f2947e576'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:03.239826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-d3b9c837-f1b9-4373-abcc-5c013cf834f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '603e936d370f4c5d98f0216f2947e576'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:03.240674 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-d3b9c837-f1b9-4373-abcc-5c013cf834f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '603e936d370f4c5d98f0216f2947e576'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:03.248316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89795a-6536-4aae-8c14-386066b38d7d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:03.253737 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2caac1-7625-4de0-9aa9-c06fb46cc0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:03.254307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-d3b9c837-f1b9-4373-abcc-5c013cf834f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '603e936d370f4c5d98f0216f2947e576'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:03.254426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2caac1-7625-4de0-9aa9-c06fb46cc0ed admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:03.254631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2caac1-7625-4de0-9aa9-c06fb46cc0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:03.254910 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2caac1-7625-4de0-9aa9-c06fb46cc0ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:03.255731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2caac1-7625-4de0-9aa9-c06fb46cc0ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:03.257923 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2caac1-7625-4de0-9aa9-c06fb46cc0ed admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:03.270968 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-d3b9c837-f1b9-4373-abcc-5c013cf834f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '603e936d370f4c5d98f0216f2947e576'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:03.287382 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 918/1853] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:24:03.289284 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2caac1-7625-4de0-9aa9-c06fb46cc0ed admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:03.291638 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1a2caac1-7625-4de0-9aa9-c06fb46cc0ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:03.304291 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 936/1854] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:03 2022] DELETE /identity/v3/users/603e936d370f4c5d98f0216f2947e576 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:03.321019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2caac1-7625-4de0-9aa9-c06fb46cc0ed admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:03.366243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b9c837-f1b9-4373-abcc-5c013cf834f5 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:03.368133 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-d3b9c837-f1b9-4373-abcc-5c013cf834f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:03.370286 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:03.468334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b9c837-f1b9-4373-abcc-5c013cf834f5 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:03.474257 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4b665-2501-45a7-8bee-50714b5a0540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:03.475200 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4b665-2501-45a7-8bee-50714b5a0540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:03.475623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4b665-2501-45a7-8bee-50714b5a0540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:03.476360 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4b665-2501-45a7-8bee-50714b5a0540 admin admin] PATH_INFO: `/v3/projects/f1b67b20446b4ceebe55e9aab09f8655` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:03.477542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220db9c5-3320-470f-8117-7acf59de041a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:03.478518 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4b665-2501-45a7-8bee-50714b5a0540 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1b67b20446b4ceebe55e9aab09f8655)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:03.478875 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220db9c5-3320-470f-8117-7acf59de041a admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:03.479213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220db9c5-3320-470f-8117-7acf59de041a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:03.479495 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220db9c5-3320-470f-8117-7acf59de041a admin admin] PATH_INFO: `/v3/projects/b2a480dfbca74a1c8b15df44baf56665/users/4eeee299e8c0430493f53bc98c0037fa/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:03.481452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4b665-2501-45a7-8bee-50714b5a0540 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:03.481729 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220db9c5-3320-470f-8117-7acf59de041a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a480dfbca74a1c8b15df44baf56665, user_id=4eeee299e8c0430493f53bc98c0037fa, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:03.484802 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220db9c5-3320-470f-8117-7acf59de041a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:03.500235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 919/1855] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:03 2022] PUT /identity/v3/projects/b2a480dfbca74a1c8b15df44baf56665/users/4eeee299e8c0430493f53bc98c0037fa/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:03.511494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1fe22-8192-4589-b9ca-0030623087e5 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:03.511813 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1fe22-8192-4589-b9ca-0030623087e5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:03.512071 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1fe22-8192-4589-b9ca-0030623087e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:03.569669 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:03.573217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-82b1fe22-8192-4589-b9ca-0030623087e5 None None] MFA Rules not processed for user `4eeee299e8c0430493f53bc98c0037fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:03.575379 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-82b1fe22-8192-4589-b9ca-0030623087e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:03.584269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b9c4b665-2501-45a7-8bee-50714b5a0540 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1b67b20446b4ceebe55e9aab09f8655', 'request_id': 'req-b9c4b665-2501-45a7-8bee-50714b5a0540'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:03.592527 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b9c4b665-2501-45a7-8bee-50714b5a0540 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1b67b20446b4ceebe55e9aab09f8655', 'request_id': 'req-b9c4b665-2501-45a7-8bee-50714b5a0540'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:03.599517 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b9c4b665-2501-45a7-8bee-50714b5a0540 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1b67b20446b4ceebe55e9aab09f8655', 'request_id': 'req-b9c4b665-2501-45a7-8bee-50714b5a0540'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:03.614609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b9c4b665-2501-45a7-8bee-50714b5a0540 admin admin] The token cache is being invalidate because project f1b67b20446b4ceebe55e9aab09f8655 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:03.614897 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b9c4b665-2501-45a7-8bee-50714b5a0540 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:03.615555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b9c4b665-2501-45a7-8bee-50714b5a0540 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:03.621871 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 937/1856] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:03 2022] DELETE /identity/v3/projects/f1b67b20446b4ceebe55e9aab09f8655 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:03.623299 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79ff38-a2dc-4171-8587-d4844c4f228a None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:03.623866 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79ff38-a2dc-4171-8587-d4844c4f228a None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:03.624639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79ff38-a2dc-4171-8587-d4844c4f228a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:03.646350 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-6d79ff38-a2dc-4171-8587-d4844c4f228a None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:03.647288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-6d79ff38-a2dc-4171-8587-d4844c4f228a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:03.663626 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 920/1857] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:03 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:03.676197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1fe22-8192-4589-b9ca-0030623087e5 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:03.678227 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-82b1fe22-8192-4589-b9ca-0030623087e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:03.742362 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 938/1858] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:24:03.742362 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79ff38-a2dc-4171-8587-d4844c4f228a None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:03.744881 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-6d79ff38-a2dc-4171-8587-d4844c4f228a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:03.753926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1fe22-8192-4589-b9ca-0030623087e5 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:03.759645 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5dd1d-3b8a-400e-978d-bc12bc4b9c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:03.760368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5dd1d-3b8a-400e-978d-bc12bc4b9c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:03.760656 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5dd1d-3b8a-400e-978d-bc12bc4b9c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:03.761486 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5dd1d-3b8a-400e-978d-bc12bc4b9c54 admin admin] PATH_INFO: `/v3/projects/c8a65efaac354355bc6659826136f3cb` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:03.762782 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5dd1d-3b8a-400e-978d-bc12bc4b9c54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8a65efaac354355bc6659826136f3cb)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:03.764791 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5dd1d-3b8a-400e-978d-bc12bc4b9c54 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:03.802661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79ff38-a2dc-4171-8587-d4844c4f228a None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:03.803195 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:03.814471 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-01c5dd1d-3b8a-400e-978d-bc12bc4b9c54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8a65efaac354355bc6659826136f3cb', 'request_id': 'req-01c5dd1d-3b8a-400e-978d-bc12bc4b9c54'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:03.820940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-01c5dd1d-3b8a-400e-978d-bc12bc4b9c54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8a65efaac354355bc6659826136f3cb', 'request_id': 'req-01c5dd1d-3b8a-400e-978d-bc12bc4b9c54'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:03.827143 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-01c5dd1d-3b8a-400e-978d-bc12bc4b9c54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8a65efaac354355bc6659826136f3cb', 'request_id': 'req-01c5dd1d-3b8a-400e-978d-bc12bc4b9c54'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:03.840817 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-01c5dd1d-3b8a-400e-978d-bc12bc4b9c54 admin admin] The token cache is being invalidate because project c8a65efaac354355bc6659826136f3cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:03.841116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-01c5dd1d-3b8a-400e-978d-bc12bc4b9c54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:03.841757 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-01c5dd1d-3b8a-400e-978d-bc12bc4b9c54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:03.843566 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 921/1859] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:03 2022] DELETE /identity/v3/projects/c8a65efaac354355bc6659826136f3cb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:03.844805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b1c39-cb7e-4b18-ab41-8974af721f26 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:03.845090 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b1c39-cb7e-4b18-ab41-8974af721f26 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:03.845363 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b1c39-cb7e-4b18-ab41-8974af721f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:03.856081 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a11ff-238a-400e-b433-ad7707a4f7b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:03.856690 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a11ff-238a-400e-b433-ad7707a4f7b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:03.856971 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a11ff-238a-400e-b433-ad7707a4f7b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:03.857244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a11ff-238a-400e-b433-ad7707a4f7b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:03.858972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-da3a11ff-238a-400e-b433-ad7707a4f7b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:03.865905 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a89b1c39-cb7e-4b18-ab41-8974af721f26 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:03.867106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a89b1c39-cb7e-4b18-ab41-8974af721f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:03.894452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:03.925387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a11ff-238a-400e-b433-ad7707a4f7b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:03.926819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a11ff-238a-400e-b433-ad7707a4f7b2 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:03.931486 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 922/1860] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:03.934062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe7cf7-e63d-4bc4-9971-100e30b9379a None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:03.934347 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe7cf7-e63d-4bc4-9971-100e30b9379a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:03.934653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe7cf7-e63d-4bc4-9971-100e30b9379a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:03.938337 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 939/1861] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:03 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:03.939211 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a11ff-238a-400e-b433-ad7707a4f7b2 service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:03.950186 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a11ff-238a-400e-b433-ad7707a4f7b2 service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:03.954588 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-93fe7cf7-e63d-4bc4-9971-100e30b9379a None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:03.955824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-93fe7cf7-e63d-4bc4-9971-100e30b9379a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:03.956278 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a919b0-aa36-4550-8c7f-1adeb2c1950b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:03.956883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a919b0-aa36-4550-8c7f-1adeb2c1950b admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:03.957162 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a919b0-aa36-4550-8c7f-1adeb2c1950b admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:03.957436 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a919b0-aa36-4550-8c7f-1adeb2c1950b admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:03.958257 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a919b0-aa36-4550-8c7f-1adeb2c1950b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:03.958549 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a919b0-aa36-4550-8c7f-1adeb2c1950b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:03.960599 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a919b0-aa36-4550-8c7f-1adeb2c1950b admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:03.965873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 923/1862] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:03.974097 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01c1ae-c1e3-429c-8f75-fc487a1d4c69 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:03.974325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01c1ae-c1e3-429c-8f75-fc487a1d4c69 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:03.974595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01c1ae-c1e3-429c-8f75-fc487a1d4c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:03.977562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 940/1863] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:03.979103 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a919b0-aa36-4550-8c7f-1adeb2c1950b admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:03.981466 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-44a919b0-aa36-4550-8c7f-1adeb2c1950b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:03.994207 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-2f01c1ae-c1e3-429c-8f75-fc487a1d4c69 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:03.994366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a919b0-aa36-4550-8c7f-1adeb2c1950b admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:03.995398 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2f01c1ae-c1e3-429c-8f75-fc487a1d4c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:03.998813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7749333-94e8-4783-905a-958ce33d37d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:03.999419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7749333-94e8-4783-905a-958ce33d37d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:03.999707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7749333-94e8-4783-905a-958ce33d37d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:04.000017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7749333-94e8-4783-905a-958ce33d37d1 admin admin] PATH_INFO: `/v3/users/3369f0db545d449cbfb8d187efab5675` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:04.001172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7749333-94e8-4783-905a-958ce33d37d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3369f0db545d449cbfb8d187efab5675)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:04.003094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7749333-94e8-4783-905a-958ce33d37d1 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:04.003515 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 924/1864] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:04.004314 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01c1ae-c1e3-429c-8f75-fc487a1d4c69 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:04.013365 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01c1ae-c1e3-429c-8f75-fc487a1d4c69 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:04.017870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb53b1-da02-47a3-98f2-5cacce0fcf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:04.018444 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb53b1-da02-47a3-98f2-5cacce0fcf0c admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:04.018719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb53b1-da02-47a3-98f2-5cacce0fcf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:04.019008 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb53b1-da02-47a3-98f2-5cacce0fcf0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:04.019606 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb53b1-da02-47a3-98f2-5cacce0fcf0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:04.021510 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb53b1-da02-47a3-98f2-5cacce0fcf0c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:04.052002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 925/1865] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:04 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:04.052828 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb53b1-da02-47a3-98f2-5cacce0fcf0c admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:04.055347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a7749333-94e8-4783-905a-958ce33d37d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3369f0db545d449cbfb8d187efab5675'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:04.056284 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a7749333-94e8-4783-905a-958ce33d37d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3369f0db545d449cbfb8d187efab5675'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:04.056909 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a7749333-94e8-4783-905a-958ce33d37d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3369f0db545d449cbfb8d187efab5675'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:04.064079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a7749333-94e8-4783-905a-958ce33d37d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3369f0db545d449cbfb8d187efab5675'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:04.077123 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a7749333-94e8-4783-905a-958ce33d37d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3369f0db545d449cbfb8d187efab5675'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:04.092069 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 941/1866] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:03 2022] DELETE /identity/v3/users/3369f0db545d449cbfb8d187efab5675 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:04.092867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7749333-94e8-4783-905a-958ce33d37d1 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:04.094207 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a7749333-94e8-4783-905a-958ce33d37d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:04.122761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb53b1-da02-47a3-98f2-5cacce0fcf0c admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:04.159159 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7749333-94e8-4783-905a-958ce33d37d1 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:04.162906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:04.168009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ccccd-0c03-4174-9b46-d72e6a534c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:04.168778 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ccccd-0c03-4174-9b46-d72e6a534c94 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:04.169064 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ccccd-0c03-4174-9b46-d72e6a534c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:04.169367 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ccccd-0c03-4174-9b46-d72e6a534c94 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:04.170009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ccccd-0c03-4174-9b46-d72e6a534c94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:04.172249 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ccccd-0c03-4174-9b46-d72e6a534c94 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:04.190524 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608cc56-ed62-4271-bc20-fa5241a7d847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:04.191134 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608cc56-ed62-4271-bc20-fa5241a7d847 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:04.191414 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608cc56-ed62-4271-bc20-fa5241a7d847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:04.191692 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608cc56-ed62-4271-bc20-fa5241a7d847 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:04.192363 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608cc56-ed62-4271-bc20-fa5241a7d847 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:04.192664 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608cc56-ed62-4271-bc20-fa5241a7d847 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:04.194273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608cc56-ed62-4271-bc20-fa5241a7d847 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:04.194838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 942/1867] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:04 2022] POST /identity/v3/users => generated 282 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:04.196508 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfee6e-2f82-4e4f-b96e-f1e642f60ce4 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:04.198550 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfee6e-2f82-4e4f-b96e-f1e642f60ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:04.198819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfee6e-2f82-4e4f-b96e-f1e642f60ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:04.211171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 926/1868] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:04.211171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608cc56-ed62-4271-bc20-fa5241a7d847 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:04.221388 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608cc56-ed62-4271-bc20-fa5241a7d847 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:04.225477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-e2dfee6e-2f82-4e4f-b96e-f1e642f60ce4 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:04.226456 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ce6c3-3758-4cba-be67-0a0596775bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:04.226989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ce6c3-3758-4cba-be67-0a0596775bef admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:04.227206 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e2dfee6e-2f82-4e4f-b96e-f1e642f60ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:04.227298 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ce6c3-3758-4cba-be67-0a0596775bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:04.227530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ce6c3-3758-4cba-be67-0a0596775bef admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:04.228949 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ce6c3-3758-4cba-be67-0a0596775bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:04.230556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ce6c3-3758-4cba-be67-0a0596775bef admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:04.237003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 943/1869] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:04.241143 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfee6e-2f82-4e4f-b96e-f1e642f60ce4 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:04.242663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 927/1870] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:04.243386 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e2dfee6e-2f82-4e4f-b96e-f1e642f60ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:04.248051 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ce6c3-3758-4cba-be67-0a0596775bef admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:04.257361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfee6e-2f82-4e4f-b96e-f1e642f60ce4 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:04.258671 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ce6c3-3758-4cba-be67-0a0596775bef admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:04.263380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccca38-9204-4513-ae80-fc4520d8599a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:04.264106 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccca38-9204-4513-ae80-fc4520d8599a admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:04.264512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccca38-9204-4513-ae80-fc4520d8599a admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:04.264953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccca38-9204-4513-ae80-fc4520d8599a admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:04.265745 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ccca38-9204-4513-ae80-fc4520d8599a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:04.267825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ccca38-9204-4513-ae80-fc4520d8599a admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:04.268686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02cb00-c92b-4916-8750-4be8c8c27ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:04.269505 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02cb00-c92b-4916-8750-4be8c8c27ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:04.269859 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02cb00-c92b-4916-8750-4be8c8c27ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:04.270201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02cb00-c92b-4916-8750-4be8c8c27ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:04.270979 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be02cb00-c92b-4916-8750-4be8c8c27ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:04.272778 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be02cb00-c92b-4916-8750-4be8c8c27ae4 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:04.281128 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:04.286922 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 928/1871] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:24:04.289256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02cb00-c92b-4916-8750-4be8c8c27ae4 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:04.291454 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-be02cb00-c92b-4916-8750-4be8c8c27ae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:04.314410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 944/1872] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:04 2022] POST /identity/v3/projects => generated 389 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:04.315347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccca38-9204-4513-ae80-fc4520d8599a admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:04.366000 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be02cb00-c92b-4916-8750-4be8c8c27ae4 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:04.402666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccca38-9204-4513-ae80-fc4520d8599a admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:04.408813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9173986-7bd4-418e-99c3-e5ea930c759a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:04.410509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9173986-7bd4-418e-99c3-e5ea930c759a admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:04.410509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9173986-7bd4-418e-99c3-e5ea930c759a admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:04.410714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9173986-7bd4-418e-99c3-e5ea930c759a admin admin] PATH_INFO: `/v3/projects/330f59a1416f45508637bd01631cb14c/users/e4b371a1fc944ad2a24b960e81d1cc30/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:04.421609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f55267-2601-4caa-bc80-86dff6f97e20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:04.422343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f55267-2601-4caa-bc80-86dff6f97e20 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:04.422711 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f55267-2601-4caa-bc80-86dff6f97e20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:04.423057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f55267-2601-4caa-bc80-86dff6f97e20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:04.443878 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9173986-7bd4-418e-99c3-e5ea930c759a admin admin] RBAC: Authorizing `identity:create_grant(project_id=330f59a1416f45508637bd01631cb14c, user_id=e4b371a1fc944ad2a24b960e81d1cc30, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:04.445629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f55267-2601-4caa-bc80-86dff6f97e20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:04.448299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f55267-2601-4caa-bc80-86dff6f97e20 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:04.451657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9173986-7bd4-418e-99c3-e5ea930c759a admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:04.489876 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 945/1873] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:04 2022] PUT /identity/v3/projects/330f59a1416f45508637bd01631cb14c/users/e4b371a1fc944ad2a24b960e81d1cc30/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:04.489876 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9173986-7bd4-418e-99c3-e5ea930c759a admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:04.490164 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 929/1874] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:04.491550 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829cd05-f10a-4ebd-ab3a-3ea79875753f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:04.491985 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829cd05-f10a-4ebd-ab3a-3ea79875753f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:04.492269 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829cd05-f10a-4ebd-ab3a-3ea79875753f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:04.544870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-7829cd05-f10a-4ebd-ab3a-3ea79875753f None None] MFA Rules not processed for user `e4b371a1fc944ad2a24b960e81d1cc30`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:04.546805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7829cd05-f10a-4ebd-ab3a-3ea79875753f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:04.612043 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9173986-7bd4-418e-99c3-e5ea930c759a admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:04.638945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d74fc-5181-47e6-b210-fc3b64aac43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:04.641911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d74fc-5181-47e6-b210-fc3b64aac43d admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:04.642195 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d74fc-5181-47e6-b210-fc3b64aac43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:04.642462 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d74fc-5181-47e6-b210-fc3b64aac43d admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:04.643105 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d74fc-5181-47e6-b210-fc3b64aac43d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:04.649495 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d74fc-5181-47e6-b210-fc3b64aac43d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:04.710408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:04.718043 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 946/1875] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:04 2022] POST /identity/v3/users => generated 295 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:04.719023 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d74fc-5181-47e6-b210-fc3b64aac43d admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:04.724703 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 930/1876] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:24:04.725670 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829cd05-f10a-4ebd-ab3a-3ea79875753f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:04.738247 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829cd05-f10a-4ebd-ab3a-3ea79875753f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:04.745964 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc902947-99e3-4037-86ba-4bdb5b7b5faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:04.746632 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc902947-99e3-4037-86ba-4bdb5b7b5faf admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:04.746908 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc902947-99e3-4037-86ba-4bdb5b7b5faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:04.747181 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc902947-99e3-4037-86ba-4bdb5b7b5faf admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:04.748216 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc902947-99e3-4037-86ba-4bdb5b7b5faf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:04.751506 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc902947-99e3-4037-86ba-4bdb5b7b5faf admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:04.803902 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 931/1877] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:04 2022] POST /identity/v3/projects => generated 365 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:04.803902 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc902947-99e3-4037-86ba-4bdb5b7b5faf admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:04.811191 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d74fc-5181-47e6-b210-fc3b64aac43d admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:04.907265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc902947-99e3-4037-86ba-4bdb5b7b5faf admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:04.923392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edd6d9-ea29-4b7e-b55f-f20879868edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:04.926412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edd6d9-ea29-4b7e-b55f-f20879868edc admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:04.926695 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edd6d9-ea29-4b7e-b55f-f20879868edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:04.926916 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edd6d9-ea29-4b7e-b55f-f20879868edc admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:04.927670 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6edd6d9-ea29-4b7e-b55f-f20879868edc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:04.931391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6edd6d9-ea29-4b7e-b55f-f20879868edc admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:04.933845 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7dec60-4ded-4acb-8d5c-527159b2df8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:04.934485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7dec60-4ded-4acb-8d5c-527159b2df8f admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:04.934747 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7dec60-4ded-4acb-8d5c-527159b2df8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:04.935016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7dec60-4ded-4acb-8d5c-527159b2df8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:04.935660 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7dec60-4ded-4acb-8d5c-527159b2df8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:04.937321 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7dec60-4ded-4acb-8d5c-527159b2df8f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:04.952681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:04.966544 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 947/1878] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:04.969612 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7dec60-4ded-4acb-8d5c-527159b2df8f admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:04.983967 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7dec60-4ded-4acb-8d5c-527159b2df8f admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:04.993506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcf149-b60a-4afc-91d4-18143e7ec132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:04.994232 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcf149-b60a-4afc-91d4-18143e7ec132 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:04.994659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcf149-b60a-4afc-91d4-18143e7ec132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:04.995035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcf149-b60a-4afc-91d4-18143e7ec132 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:04.995869 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddcf149-b60a-4afc-91d4-18143e7ec132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:04.997984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddcf149-b60a-4afc-91d4-18143e7ec132 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:05.004640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 932/1879] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:04 2022] POST /identity/v3/users => generated 289 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:05.011465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 948/1880] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:05.012469 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcf149-b60a-4afc-91d4-18143e7ec132 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:05.018543 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edd6d9-ea29-4b7e-b55f-f20879868edc admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:05.025077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcf149-b60a-4afc-91d4-18143e7ec132 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:05.032875 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48397dc0-5796-4937-8d37-6acb73915b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:05.033591 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48397dc0-5796-4937-8d37-6acb73915b34 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:05.033981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48397dc0-5796-4937-8d37-6acb73915b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:05.034865 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48397dc0-5796-4937-8d37-6acb73915b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:05.034978 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48397dc0-5796-4937-8d37-6acb73915b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:05.036919 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48397dc0-5796-4937-8d37-6acb73915b34 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:05.037251 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edd6d9-ea29-4b7e-b55f-f20879868edc admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:05.042964 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433d166-f914-4f6e-8143-f8df2417ed22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:05.043811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433d166-f914-4f6e-8143-f8df2417ed22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:05.043997 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433d166-f914-4f6e-8143-f8df2417ed22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:05.044323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433d166-f914-4f6e-8143-f8df2417ed22 admin admin] PATH_INFO: `/v3/projects/ff18b88445a14bad93b5e2e9fe1e78c4/users/cd267e19e72440678a63c6889abaa877/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:05.064947 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 949/1881] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:05.065331 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433d166-f914-4f6e-8143-f8df2417ed22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff18b88445a14bad93b5e2e9fe1e78c4, user_id=cd267e19e72440678a63c6889abaa877, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:05.068958 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433d166-f914-4f6e-8143-f8df2417ed22 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:05.074316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48397dc0-5796-4937-8d37-6acb73915b34 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:05.092853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 933/1882] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:05 2022] PUT /identity/v3/projects/ff18b88445a14bad93b5e2e9fe1e78c4/users/cd267e19e72440678a63c6889abaa877/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:05.102823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c392038-a631-4808-9c66-3219828be80c None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:05.103110 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c392038-a631-4808-9c66-3219828be80c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:05.103388 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c392038-a631-4808-9c66-3219828be80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:05.105204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48397dc0-5796-4937-8d37-6acb73915b34 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:05.141570 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6c392038-a631-4808-9c66-3219828be80c None None] MFA Rules not processed for user `cd267e19e72440678a63c6889abaa877`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:05.142805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6c392038-a631-4808-9c66-3219828be80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:05.207666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45937c-f5ad-4839-965b-a0f4e98736f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:05.208388 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45937c-f5ad-4839-965b-a0f4e98736f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:05.209251 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45937c-f5ad-4839-965b-a0f4e98736f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:05.209251 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45937c-f5ad-4839-965b-a0f4e98736f9 admin admin] PATH_INFO: `/v3/projects/90893c26b4b04527a58a61f894d91396/users/58142d0708e349298ca9cd5f069be3bd/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:05.244666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45937c-f5ad-4839-965b-a0f4e98736f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90893c26b4b04527a58a61f894d91396, user_id=58142d0708e349298ca9cd5f069be3bd, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:05.247853 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45937c-f5ad-4839-965b-a0f4e98736f9 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:05.274216 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 950/1883] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:05 2022] PUT /identity/v3/projects/90893c26b4b04527a58a61f894d91396/users/58142d0708e349298ca9cd5f069be3bd/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:05.278873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45937c-f5ad-4839-965b-a0f4e98736f9 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:05.310142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 934/1884] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:05.321108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c392038-a631-4808-9c66-3219828be80c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:05.348842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c392038-a631-4808-9c66-3219828be80c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:05.405991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:05.436603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45937c-f5ad-4839-965b-a0f4e98736f9 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:05.441836 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f47afb-0d47-465c-acd8-db3e1d0448f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:05.442472 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f47afb-0d47-465c-acd8-db3e1d0448f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:05.442730 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f47afb-0d47-465c-acd8-db3e1d0448f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:05.443089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f47afb-0d47-465c-acd8-db3e1d0448f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:05.443647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f47afb-0d47-465c-acd8-db3e1d0448f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:05.445292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f47afb-0d47-465c-acd8-db3e1d0448f6 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:05.445643 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd091d-d4e3-4d14-a6df-902ca13458b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:05.446319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd091d-d4e3-4d14-a6df-902ca13458b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:05.446750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd091d-d4e3-4d14-a6df-902ca13458b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:05.447099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd091d-d4e3-4d14-a6df-902ca13458b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:05.447844 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd091d-d4e3-4d14-a6df-902ca13458b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:05.450754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd091d-d4e3-4d14-a6df-902ca13458b5 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:05.457638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 951/1885] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:05.459216 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f47afb-0d47-465c-acd8-db3e1d0448f6 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:05.463920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b6f47afb-0d47-465c-acd8-db3e1d0448f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:05.479929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f47afb-0d47-465c-acd8-db3e1d0448f6 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:05.485826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bacdf2-51a1-49f6-8ad7-5a3e250d6ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:05.486661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bacdf2-51a1-49f6-8ad7-5a3e250d6ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:05.487090 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bacdf2-51a1-49f6-8ad7-5a3e250d6ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:05.487545 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bacdf2-51a1-49f6-8ad7-5a3e250d6ece admin admin] PATH_INFO: `/v3/projects/14d8e11e839c4d88b7612b826619ecc6` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:05.489307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bacdf2-51a1-49f6-8ad7-5a3e250d6ece admin admin] RBAC: Authorizing `identity:delete_project(project_id=14d8e11e839c4d88b7612b826619ecc6)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:05.492220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bacdf2-51a1-49f6-8ad7-5a3e250d6ece admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:05.531824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 935/1886] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:05 2022] POST /identity/v3/projects => generated 387 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:05.534212 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd091d-d4e3-4d14-a6df-902ca13458b5 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:05.558400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e2bacdf2-51a1-49f6-8ad7-5a3e250d6ece admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14d8e11e839c4d88b7612b826619ecc6', 'request_id': 'req-e2bacdf2-51a1-49f6-8ad7-5a3e250d6ece'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:05.561117 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:05.569244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e2bacdf2-51a1-49f6-8ad7-5a3e250d6ece admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14d8e11e839c4d88b7612b826619ecc6', 'request_id': 'req-e2bacdf2-51a1-49f6-8ad7-5a3e250d6ece'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:05.579675 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e2bacdf2-51a1-49f6-8ad7-5a3e250d6ece admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14d8e11e839c4d88b7612b826619ecc6', 'request_id': 'req-e2bacdf2-51a1-49f6-8ad7-5a3e250d6ece'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:05.603223 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e2bacdf2-51a1-49f6-8ad7-5a3e250d6ece admin admin] The token cache is being invalidate because project 14d8e11e839c4d88b7612b826619ecc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:05.604115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e2bacdf2-51a1-49f6-8ad7-5a3e250d6ece admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:05.605510 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e2bacdf2-51a1-49f6-8ad7-5a3e250d6ece admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:05.607127 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 952/1887] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:05 2022] DELETE /identity/v3/projects/14d8e11e839c4d88b7612b826619ecc6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:05.607652 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bacdf2-51a1-49f6-8ad7-5a3e250d6ece admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:05.611966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e2bacdf2-51a1-49f6-8ad7-5a3e250d6ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:05.692344 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd091d-d4e3-4d14-a6df-902ca13458b5 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:05.760400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bacdf2-51a1-49f6-8ad7-5a3e250d6ece admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:05.766440 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e5127-87e0-42e7-97ba-f3440870eef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:05.767498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e5127-87e0-42e7-97ba-f3440870eef8 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:05.767832 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e5127-87e0-42e7-97ba-f3440870eef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:05.768103 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e5127-87e0-42e7-97ba-f3440870eef8 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:05.769008 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e5127-87e0-42e7-97ba-f3440870eef8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:05.769748 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33082231-5b40-4037-81a4-efe609850376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:05.770936 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33082231-5b40-4037-81a4-efe609850376 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:05.771046 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e5127-87e0-42e7-97ba-f3440870eef8 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:05.771533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33082231-5b40-4037-81a4-efe609850376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:05.772508 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33082231-5b40-4037-81a4-efe609850376 admin admin] PATH_INFO: `/v3/projects/90893c26b4b04527a58a61f894d91396/users/58142d0708e349298ca9cd5f069be3bd/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:05.775094 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33082231-5b40-4037-81a4-efe609850376 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90893c26b4b04527a58a61f894d91396, user_id=58142d0708e349298ca9cd5f069be3bd, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:05.779074 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33082231-5b40-4037-81a4-efe609850376 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:05.805158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 953/1889] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:05 2022] POST /identity/v3/users => generated 300 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:05.806644 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e5127-87e0-42e7-97ba-f3440870eef8 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:05.808765 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 936/1889] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:05 2022] PUT /identity/v3/projects/90893c26b4b04527a58a61f894d91396/users/58142d0708e349298ca9cd5f069be3bd/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:05.811828 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-1b2e5127-87e0-42e7-97ba-f3440870eef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:05.814526 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cadd3-a6a3-4c25-8dcf-cca2c6e9ae6f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:05.815287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cadd3-a6a3-4c25-8dcf-cca2c6e9ae6f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:05.815585 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cadd3-a6a3-4c25-8dcf-cca2c6e9ae6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:05.835667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e5127-87e0-42e7-97ba-f3440870eef8 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:05.876154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-5f3cadd3-a6a3-4c25-8dcf-cca2c6e9ae6f None None] MFA Rules not processed for user `58142d0708e349298ca9cd5f069be3bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:05.879222 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5f3cadd3-a6a3-4c25-8dcf-cca2c6e9ae6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:05.939628 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36efae59-de04-489b-ac05-5b8dc43a5120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:05.940406 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efae59-de04-489b-ac05-5b8dc43a5120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:05.940711 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efae59-de04-489b-ac05-5b8dc43a5120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:05.941005 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efae59-de04-489b-ac05-5b8dc43a5120 admin admin] PATH_INFO: `/v3/users/4eeee299e8c0430493f53bc98c0037fa` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:05.942268 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36efae59-de04-489b-ac05-5b8dc43a5120 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eeee299e8c0430493f53bc98c0037fa)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:05.944221 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36efae59-de04-489b-ac05-5b8dc43a5120 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:05.944401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:05.977717 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 937/1890] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:05.977717 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3cadd3-a6a3-4c25-8dcf-cca2c6e9ae6f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:05.985710 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3cadd3-a6a3-4c25-8dcf-cca2c6e9ae6f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:05.991785 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada9881-330e-4f16-8fa5-8e4f1a6ba57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:05.992459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada9881-330e-4f16-8fa5-8e4f1a6ba57c admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:05.992795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada9881-330e-4f16-8fa5-8e4f1a6ba57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:05.993094 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada9881-330e-4f16-8fa5-8e4f1a6ba57c admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:05.994034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada9881-330e-4f16-8fa5-8e4f1a6ba57c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:05.995914 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada9881-330e-4f16-8fa5-8e4f1a6ba57c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:06.006983 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 938/1891] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:06.007755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada9881-330e-4f16-8fa5-8e4f1a6ba57c admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:06.010122 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-dada9881-330e-4f16-8fa5-8e4f1a6ba57c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:06.014365 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-36efae59-de04-489b-ac05-5b8dc43a5120 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eeee299e8c0430493f53bc98c0037fa'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:06.015057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-36efae59-de04-489b-ac05-5b8dc43a5120 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eeee299e8c0430493f53bc98c0037fa'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:06.015749 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-36efae59-de04-489b-ac05-5b8dc43a5120 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eeee299e8c0430493f53bc98c0037fa'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:06.023469 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-36efae59-de04-489b-ac05-5b8dc43a5120 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eeee299e8c0430493f53bc98c0037fa'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:06.035931 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:06.037157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-36efae59-de04-489b-ac05-5b8dc43a5120 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eeee299e8c0430493f53bc98c0037fa'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:06.057528 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 954/1892] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:05 2022] DELETE /identity/v3/users/4eeee299e8c0430493f53bc98c0037fa => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:06.058523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36efae59-de04-489b-ac05-5b8dc43a5120 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:06.059631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-36efae59-de04-489b-ac05-5b8dc43a5120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:06.084905 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada9881-330e-4f16-8fa5-8e4f1a6ba57c admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:06.144850 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36efae59-de04-489b-ac05-5b8dc43a5120 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:06.152024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc861d66-4cad-41b3-82c2-5b3b279a252a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:06.152638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc861d66-4cad-41b3-82c2-5b3b279a252a admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:06.152925 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc861d66-4cad-41b3-82c2-5b3b279a252a admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:06.153281 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc861d66-4cad-41b3-82c2-5b3b279a252a admin admin] PATH_INFO: `/v3/projects/2be63f32b9d540af8d8626aee59410da/users/0e2ade0c86a844f2a0fac1452f2c4fbf/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:06.172088 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124bd2e-5d5d-4e09-bf04-7dc1f7c55427 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:06.172951 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124bd2e-5d5d-4e09-bf04-7dc1f7c55427 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:06.173326 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124bd2e-5d5d-4e09-bf04-7dc1f7c55427 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:06.173577 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124bd2e-5d5d-4e09-bf04-7dc1f7c55427 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:06.175447 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0124bd2e-5d5d-4e09-bf04-7dc1f7c55427 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:06.176093 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc861d66-4cad-41b3-82c2-5b3b279a252a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2be63f32b9d540af8d8626aee59410da, user_id=0e2ade0c86a844f2a0fac1452f2c4fbf, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:06.181771 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc861d66-4cad-41b3-82c2-5b3b279a252a admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:06.210017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 955/1893] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:06 2022] PUT /identity/v3/projects/2be63f32b9d540af8d8626aee59410da/users/0e2ade0c86a844f2a0fac1452f2c4fbf/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:06.214630 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc861d66-4cad-41b3-82c2-5b3b279a252a admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:06.264538 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124bd2e-5d5d-4e09-bf04-7dc1f7c55427 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:06.266385 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124bd2e-5d5d-4e09-bf04-7dc1f7c55427 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:06.279183 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 939/1894] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:06 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:06.310947 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc861d66-4cad-41b3-82c2-5b3b279a252a admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:06.318833 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f84c6-c8a8-48f8-a527-0cc5ed6eac12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:06.319967 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f84c6-c8a8-48f8-a527-0cc5ed6eac12 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:06.320342 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f84c6-c8a8-48f8-a527-0cc5ed6eac12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:06.320728 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f84c6-c8a8-48f8-a527-0cc5ed6eac12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:06.321528 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f84c6-c8a8-48f8-a527-0cc5ed6eac12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:06.323483 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f84c6-c8a8-48f8-a527-0cc5ed6eac12 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:06.339440 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 956/1895] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:24:06.340920 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124bd2e-5d5d-4e09-bf04-7dc1f7c55427 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:06.369011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124bd2e-5d5d-4e09-bf04-7dc1f7c55427 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:06.388960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832ed97-4095-4f3c-8812-4faa15ba55f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:06.394641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832ed97-4095-4f3c-8812-4faa15ba55f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:06.394945 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832ed97-4095-4f3c-8812-4faa15ba55f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:06.395567 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832ed97-4095-4f3c-8812-4faa15ba55f1 admin admin] PATH_INFO: `/v3/projects/2be63f32b9d540af8d8626aee59410da/users/0e2ade0c86a844f2a0fac1452f2c4fbf/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:06.398710 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5832ed97-4095-4f3c-8812-4faa15ba55f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2be63f32b9d540af8d8626aee59410da, user_id=0e2ade0c86a844f2a0fac1452f2c4fbf, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:06.403473 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5832ed97-4095-4f3c-8812-4faa15ba55f1 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:06.423255 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f84c6-c8a8-48f8-a527-0cc5ed6eac12 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:06.426160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 940/1896] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:06 2022] PUT /identity/v3/projects/2be63f32b9d540af8d8626aee59410da/users/0e2ade0c86a844f2a0fac1452f2c4fbf/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:06.426545 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-da9f84c6-c8a8-48f8-a527-0cc5ed6eac12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:06.442160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8528df4-c111-4384-a0d4-5a0cf29dec23 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:06.442508 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8528df4-c111-4384-a0d4-5a0cf29dec23 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:06.442811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8528df4-c111-4384-a0d4-5a0cf29dec23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:06.497994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d8528df4-c111-4384-a0d4-5a0cf29dec23 None None] MFA Rules not processed for user `0e2ade0c86a844f2a0fac1452f2c4fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:06.499750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d8528df4-c111-4384-a0d4-5a0cf29dec23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:06.516458 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:06.568719 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f84c6-c8a8-48f8-a527-0cc5ed6eac12 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:06.617354 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:06.625239 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 941/1897] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:06 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:06.671512 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8528df4-c111-4384-a0d4-5a0cf29dec23 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:06.673154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d8528df4-c111-4384-a0d4-5a0cf29dec23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:06.699335 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a21a0-a6a9-4d8c-b48e-fe68af41f063 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:06.700447 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a21a0-a6a9-4d8c-b48e-fe68af41f063 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:06.700883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a21a0-a6a9-4d8c-b48e-fe68af41f063 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:06.701251 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a21a0-a6a9-4d8c-b48e-fe68af41f063 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:06.704149 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c14a21a0-a6a9-4d8c-b48e-fe68af41f063 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:06.773701 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8528df4-c111-4384-a0d4-5a0cf29dec23 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:06.779003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74250a-d0e3-43e0-a8d0-cb5f25d71e3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:06.779682 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74250a-d0e3-43e0-a8d0-cb5f25d71e3e service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:06.780093 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74250a-d0e3-43e0-a8d0-cb5f25d71e3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:06.780369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74250a-d0e3-43e0-a8d0-cb5f25d71e3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:06.789890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a21a0-a6a9-4d8c-b48e-fe68af41f063 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:06.792195 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a21a0-a6a9-4d8c-b48e-fe68af41f063 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:06.793469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74250a-d0e3-43e0-a8d0-cb5f25d71e3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:06.795421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74250a-d0e3-43e0-a8d0-cb5f25d71e3e service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:06.810243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 957/1898] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:06.828188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 942/1899] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:06 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:07.157774 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a21a0-a6a9-4d8c-b48e-fe68af41f063 service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:07.160399 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c14a21a0-a6a9-4d8c-b48e-fe68af41f063 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:07.183268 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74250a-d0e3-43e0-a8d0-cb5f25d71e3e service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:07.248884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a21a0-a6a9-4d8c-b48e-fe68af41f063 service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:07.253543 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce9cf7-10e3-49b1-8504-55afa9d4a66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:07.254169 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce9cf7-10e3-49b1-8504-55afa9d4a66f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:07.254449 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce9cf7-10e3-49b1-8504-55afa9d4a66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:07.254714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce9cf7-10e3-49b1-8504-55afa9d4a66f admin admin] PATH_INFO: `/v3/users/87c4421d1dac42b796be3d32e0a747db` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:07.255997 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce9cf7-10e3-49b1-8504-55afa9d4a66f admin admin] RBAC: Authorizing `identity:delete_user(user_id=87c4421d1dac42b796be3d32e0a747db)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:07.257932 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce9cf7-10e3-49b1-8504-55afa9d4a66f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:07.262102 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74250a-d0e3-43e0-a8d0-cb5f25d71e3e service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:07.265180 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe8463-4f5d-419e-b52e-af24d6f77790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:07.265806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe8463-4f5d-419e-b52e-af24d6f77790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:07.266086 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe8463-4f5d-419e-b52e-af24d6f77790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:07.267620 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe8463-4f5d-419e-b52e-af24d6f77790 admin admin] PATH_INFO: `/v3/users/cd267e19e72440678a63c6889abaa877` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:07.267733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fe8463-4f5d-419e-b52e-af24d6f77790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd267e19e72440678a63c6889abaa877)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:07.269868 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fe8463-4f5d-419e-b52e-af24d6f77790 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:07.315500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-56ce9cf7-10e3-49b1-8504-55afa9d4a66f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87c4421d1dac42b796be3d32e0a747db'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:07.316161 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-56ce9cf7-10e3-49b1-8504-55afa9d4a66f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87c4421d1dac42b796be3d32e0a747db'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:07.317555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-56ce9cf7-10e3-49b1-8504-55afa9d4a66f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87c4421d1dac42b796be3d32e0a747db'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:07.324481 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-85fe8463-4f5d-419e-b52e-af24d6f77790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd267e19e72440678a63c6889abaa877'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:07.325513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-56ce9cf7-10e3-49b1-8504-55afa9d4a66f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87c4421d1dac42b796be3d32e0a747db'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:07.345710 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-56ce9cf7-10e3-49b1-8504-55afa9d4a66f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87c4421d1dac42b796be3d32e0a747db'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:07.346124 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-85fe8463-4f5d-419e-b52e-af24d6f77790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd267e19e72440678a63c6889abaa877'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:07.347011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-85fe8463-4f5d-419e-b52e-af24d6f77790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd267e19e72440678a63c6889abaa877'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:07.347825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-85fe8463-4f5d-419e-b52e-af24d6f77790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd267e19e72440678a63c6889abaa877'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:07.350540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:07.376865 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:07.377667 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-85fe8463-4f5d-419e-b52e-af24d6f77790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd267e19e72440678a63c6889abaa877'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:07.390736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 958/1900] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:07 2022] DELETE /identity/v3/users/87c4421d1dac42b796be3d32e0a747db => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:07.395009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ceb7e-7603-4f42-845d-7fab1a063921 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:07.395152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 943/1901] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:07 2022] DELETE /identity/v3/users/cd267e19e72440678a63c6889abaa877 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:07.395248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ceb7e-7603-4f42-845d-7fab1a063921 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:07.395248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ceb7e-7603-4f42-845d-7fab1a063921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:07.396475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9de00-8173-4dae-9241-206f62037433 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:07.396774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9de00-8173-4dae-9241-206f62037433 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:07.397079 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9de00-8173-4dae-9241-206f62037433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:07.419892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-87d9de00-8173-4dae-9241-206f62037433 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:07.421134 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-87d9de00-8173-4dae-9241-206f62037433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:07.421961 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-383ceb7e-7603-4f42-845d-7fab1a063921 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:07.424168 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-383ceb7e-7603-4f42-845d-7fab1a063921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:07.498736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 944/1902] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:07.508633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9de00-8173-4dae-9241-206f62037433 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:07.508915 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 959/1903] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:07.510366 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-87d9de00-8173-4dae-9241-206f62037433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:07.518220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ceb7e-7603-4f42-845d-7fab1a063921 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:07.519934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-383ceb7e-7603-4f42-845d-7fab1a063921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:07.606528 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ceb7e-7603-4f42-845d-7fab1a063921 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:07.611371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a35dc-dc62-48a1-9810-b348e2722cda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:07.612167 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a35dc-dc62-48a1-9810-b348e2722cda service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:07.612740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a35dc-dc62-48a1-9810-b348e2722cda service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:07.613120 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a35dc-dc62-48a1-9810-b348e2722cda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:07.621659 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9de00-8173-4dae-9241-206f62037433 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:07.627124 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71b5e5-5109-4223-a778-b5c0d55908c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:07.628784 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71b5e5-5109-4223-a778-b5c0d55908c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:07.629646 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71b5e5-5109-4223-a778-b5c0d55908c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:07.630192 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71b5e5-5109-4223-a778-b5c0d55908c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:07.633288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a35dc-dc62-48a1-9810-b348e2722cda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:07.641931 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a35dc-dc62-48a1-9810-b348e2722cda service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:07.646102 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71b5e5-5109-4223-a778-b5c0d55908c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:07.647800 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71b5e5-5109-4223-a778-b5c0d55908c6 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:07.662598 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 945/1904] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:07.678471 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 960/1905] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:24:07.684185 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71b5e5-5109-4223-a778-b5c0d55908c6 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:07.699474 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-aa71b5e5-5109-4223-a778-b5c0d55908c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:07.717396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71b5e5-5109-4223-a778-b5c0d55908c6 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:07.724032 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6cb722-aa19-4027-a747-ab82cfe27ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:07.724753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cb722-aa19-4027-a747-ab82cfe27ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:07.725107 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cb722-aa19-4027-a747-ab82cfe27ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:07.725454 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cb722-aa19-4027-a747-ab82cfe27ba7 admin admin] PATH_INFO: `/v3/projects/b2a480dfbca74a1c8b15df44baf56665` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:07.726813 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6cb722-aa19-4027-a747-ab82cfe27ba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2a480dfbca74a1c8b15df44baf56665)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:07.728895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6cb722-aa19-4027-a747-ab82cfe27ba7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:07.792597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5f6cb722-aa19-4027-a747-ab82cfe27ba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2a480dfbca74a1c8b15df44baf56665', 'request_id': 'req-5f6cb722-aa19-4027-a747-ab82cfe27ba7'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:07.803959 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5f6cb722-aa19-4027-a747-ab82cfe27ba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2a480dfbca74a1c8b15df44baf56665', 'request_id': 'req-5f6cb722-aa19-4027-a747-ab82cfe27ba7'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:07.814281 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5f6cb722-aa19-4027-a747-ab82cfe27ba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2a480dfbca74a1c8b15df44baf56665', 'request_id': 'req-5f6cb722-aa19-4027-a747-ab82cfe27ba7'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:07.831051 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:07.840317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5f6cb722-aa19-4027-a747-ab82cfe27ba7 admin admin] The token cache is being invalidate because project b2a480dfbca74a1c8b15df44baf56665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:07.840744 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5f6cb722-aa19-4027-a747-ab82cfe27ba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:07.842352 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5f6cb722-aa19-4027-a747-ab82cfe27ba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:07.846824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 946/1906] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:07 2022] DELETE /identity/v3/projects/b2a480dfbca74a1c8b15df44baf56665 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:07.851825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e657175-54f6-4f5b-af7c-f5a34631d80d None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:07.851825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e657175-54f6-4f5b-af7c-f5a34631d80d None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:07.852167 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e657175-54f6-4f5b-af7c-f5a34631d80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:07.884984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-9e657175-54f6-4f5b-af7c-f5a34631d80d None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:07.886779 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-9e657175-54f6-4f5b-af7c-f5a34631d80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:07.969851 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 961/1907] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:24:07.978421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce778b5-1c82-4730-a708-df247c8b222c None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:07.978755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce778b5-1c82-4730-a708-df247c8b222c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:07.979056 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce778b5-1c82-4730-a708-df247c8b222c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:08.011452 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-7ce778b5-1c82-4730-a708-df247c8b222c None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:08.012751 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7ce778b5-1c82-4730-a708-df247c8b222c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:08.022435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 947/1908] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:08.027250 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e657175-54f6-4f5b-af7c-f5a34631d80d None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:08.040701 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e657175-54f6-4f5b-af7c-f5a34631d80d None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:08.046267 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4cf25-0d7d-4304-a4a0-b39509ef49f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:08.047269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4cf25-0d7d-4304-a4a0-b39509ef49f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:08.047552 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4cf25-0d7d-4304-a4a0-b39509ef49f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:08.047853 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4cf25-0d7d-4304-a4a0-b39509ef49f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:08.048514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff4cf25-0d7d-4304-a4a0-b39509ef49f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:08.048817 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff4cf25-0d7d-4304-a4a0-b39509ef49f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:08.050458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff4cf25-0d7d-4304-a4a0-b39509ef49f3 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:08.067895 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 962/1909] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:08.069780 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce778b5-1c82-4730-a708-df247c8b222c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:08.105621 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce778b5-1c82-4730-a708-df247c8b222c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:08.116764 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3bad6b-82f3-4dd4-8f4a-4936e1f80596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:08.117609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bad6b-82f3-4dd4-8f4a-4936e1f80596 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:08.118123 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bad6b-82f3-4dd4-8f4a-4936e1f80596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:08.119158 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bad6b-82f3-4dd4-8f4a-4936e1f80596 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:08.120033 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bad6b-82f3-4dd4-8f4a-4936e1f80596 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:08.123239 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bad6b-82f3-4dd4-8f4a-4936e1f80596 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:08.190263 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 948/1910] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:08 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:08.191969 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4cf25-0d7d-4304-a4a0-b39509ef49f3 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:08.210113 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4cf25-0d7d-4304-a4a0-b39509ef49f3 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:08.272129 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:08.320853 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e5c2b-f2fb-46c1-af7b-399e03d7c52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:08.322086 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e5c2b-f2fb-46c1-af7b-399e03d7c52a admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:08.322372 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e5c2b-f2fb-46c1-af7b-399e03d7c52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:08.322674 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e5c2b-f2fb-46c1-af7b-399e03d7c52a admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:08.323597 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e5c2b-f2fb-46c1-af7b-399e03d7c52a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:08.325794 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e5c2b-f2fb-46c1-af7b-399e03d7c52a admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:08.357050 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 963/1911] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:08 2022] POST /identity/v3/users => generated 294 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:08.361481 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3bad6b-82f3-4dd4-8f4a-4936e1f80596 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:08.382865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3bad6b-82f3-4dd4-8f4a-4936e1f80596 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:08.388602 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b43fb-5bc2-453b-aed0-40f75d1857b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:08.389581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b43fb-5bc2-453b-aed0-40f75d1857b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:08.389865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b43fb-5bc2-453b-aed0-40f75d1857b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:08.390684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b43fb-5bc2-453b-aed0-40f75d1857b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:08.391893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b43fb-5bc2-453b-aed0-40f75d1857b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:08.393628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b43fb-5bc2-453b-aed0-40f75d1857b2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:08.407751 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 949/1912] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:08.411954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e5c2b-f2fb-46c1-af7b-399e03d7c52a admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:08.428993 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e5c2b-f2fb-46c1-af7b-399e03d7c52a admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:08.436761 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d70ca3-9c5f-460e-addc-21470ec6bf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:08.437443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d70ca3-9c5f-460e-addc-21470ec6bf3d admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:08.437731 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d70ca3-9c5f-460e-addc-21470ec6bf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:08.438057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d70ca3-9c5f-460e-addc-21470ec6bf3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:08.438828 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d70ca3-9c5f-460e-addc-21470ec6bf3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:08.441456 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d70ca3-9c5f-460e-addc-21470ec6bf3d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:08.460389 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 964/1913] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:08.464774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b43fb-5bc2-453b-aed0-40f75d1857b2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:08.485814 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b43fb-5bc2-453b-aed0-40f75d1857b2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:08.492579 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d382b4d-8e1e-42d2-86d5-70ce04e86607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:08.493540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d382b4d-8e1e-42d2-86d5-70ce04e86607 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:08.493943 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d382b4d-8e1e-42d2-86d5-70ce04e86607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:08.494393 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d382b4d-8e1e-42d2-86d5-70ce04e86607 admin admin] PATH_INFO: `/v3/projects/aa0a0b92bf1243c1821447758325bcf9/users/1cd9db584be246bebcc6a3cb494aacb6/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:08.516708 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d382b4d-8e1e-42d2-86d5-70ce04e86607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa0a0b92bf1243c1821447758325bcf9, user_id=1cd9db584be246bebcc6a3cb494aacb6, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:08.520622 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d382b4d-8e1e-42d2-86d5-70ce04e86607 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:08.551310 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 950/1914] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:08 2022] PUT /identity/v3/projects/aa0a0b92bf1243c1821447758325bcf9/users/1cd9db584be246bebcc6a3cb494aacb6/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:08.562273 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcbd10-48e9-4ba7-84cd-3624c9c217f9 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:08.562470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcbd10-48e9-4ba7-84cd-3624c9c217f9 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:08.562746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcbd10-48e9-4ba7-84cd-3624c9c217f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:08.623903 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-c8bcbd10-48e9-4ba7-84cd-3624c9c217f9 None None] MFA Rules not processed for user `1cd9db584be246bebcc6a3cb494aacb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:08.626981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c8bcbd10-48e9-4ba7-84cd-3624c9c217f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:08.767146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 965/1915] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:08.768377 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d382b4d-8e1e-42d2-86d5-70ce04e86607 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:08.869751 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d382b4d-8e1e-42d2-86d5-70ce04e86607 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:08.877890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d994625-46bc-42cb-bbaa-fb30c47c65fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:08.878608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d994625-46bc-42cb-bbaa-fb30c47c65fe admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:08.879003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d994625-46bc-42cb-bbaa-fb30c47c65fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:08.879338 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d994625-46bc-42cb-bbaa-fb30c47c65fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:08.880339 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d994625-46bc-42cb-bbaa-fb30c47c65fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:08.883165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d994625-46bc-42cb-bbaa-fb30c47c65fe admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:08.925978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 951/1916] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:08 2022] POST /identity/v3/projects => generated 387 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:08.931194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bcbd10-48e9-4ba7-84cd-3624c9c217f9 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:08.948208 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bcbd10-48e9-4ba7-84cd-3624c9c217f9 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:09.018916 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be080bc-6c59-4e90-a29a-98cbecb15345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:09.019580 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be080bc-6c59-4e90-a29a-98cbecb15345 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:09.019880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be080bc-6c59-4e90-a29a-98cbecb15345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:09.020154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be080bc-6c59-4e90-a29a-98cbecb15345 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:09.020785 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be080bc-6c59-4e90-a29a-98cbecb15345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:09.022697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be080bc-6c59-4e90-a29a-98cbecb15345 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:09.072073 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d994625-46bc-42cb-bbaa-fb30c47c65fe admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:09.073303 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 966/1917] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:08 2022] POST /identity/v3/users => generated 294 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:09.104314 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d994625-46bc-42cb-bbaa-fb30c47c65fe admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:09.114749 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e1870-cbd8-4103-9dc1-2f28c57403a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:09.115414 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e1870-cbd8-4103-9dc1-2f28c57403a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:09.116249 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e1870-cbd8-4103-9dc1-2f28c57403a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:09.116531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e1870-cbd8-4103-9dc1-2f28c57403a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:09.117562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e1870-cbd8-4103-9dc1-2f28c57403a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:09.121811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e1870-cbd8-4103-9dc1-2f28c57403a2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:09.141148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 952/1918] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:24:09.142911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be080bc-6c59-4e90-a29a-98cbecb15345 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:09.166699 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be080bc-6c59-4e90-a29a-98cbecb15345 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:09.172985 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8d309-674a-4287-86d9-f542dc65c48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:09.173662 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8d309-674a-4287-86d9-f542dc65c48d admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:09.173911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8d309-674a-4287-86d9-f542dc65c48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:09.174185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8d309-674a-4287-86d9-f542dc65c48d admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:09.174884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a8d309-674a-4287-86d9-f542dc65c48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:09.176974 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a8d309-674a-4287-86d9-f542dc65c48d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:09.177418 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e1870-cbd8-4103-9dc1-2f28c57403a2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:09.179691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-015e1870-cbd8-4103-9dc1-2f28c57403a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:09.192423 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 967/1919] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:09.196337 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8d309-674a-4287-86d9-f542dc65c48d admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:09.198766 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e1870-cbd8-4103-9dc1-2f28c57403a2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:09.205135 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6749d04-21f7-4ea0-9f80-3e80f529cad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:09.206069 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6749d04-21f7-4ea0-9f80-3e80f529cad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:09.206421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6749d04-21f7-4ea0-9f80-3e80f529cad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:09.206859 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6749d04-21f7-4ea0-9f80-3e80f529cad6 admin admin] PATH_INFO: `/v3/projects/f98ea508636e4fe49dc5fb553c10a79e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:09.208225 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6749d04-21f7-4ea0-9f80-3e80f529cad6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f98ea508636e4fe49dc5fb553c10a79e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:09.210516 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6749d04-21f7-4ea0-9f80-3e80f529cad6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:09.224994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8d309-674a-4287-86d9-f542dc65c48d admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:09.251163 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f27f8-6be7-4573-afa7-8a2e2c59404e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:09.251890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f27f8-6be7-4573-afa7-8a2e2c59404e admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:09.252172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f27f8-6be7-4573-afa7-8a2e2c59404e admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:09.253961 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f27f8-6be7-4573-afa7-8a2e2c59404e admin admin] PATH_INFO: `/v3/projects/333b3a8c680f4e54a1f0946d4b011217/users/0fe036638eab4133b31bb30076261c17/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:09.268002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:09.285727 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f27f8-6be7-4573-afa7-8a2e2c59404e admin admin] RBAC: Authorizing `identity:create_grant(project_id=333b3a8c680f4e54a1f0946d4b011217, user_id=0fe036638eab4133b31bb30076261c17, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:09.289109 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f27f8-6be7-4573-afa7-8a2e2c59404e admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:09.306292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e6749d04-21f7-4ea0-9f80-3e80f529cad6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f98ea508636e4fe49dc5fb553c10a79e', 'request_id': 'req-e6749d04-21f7-4ea0-9f80-3e80f529cad6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:09.315114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e6749d04-21f7-4ea0-9f80-3e80f529cad6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f98ea508636e4fe49dc5fb553c10a79e', 'request_id': 'req-e6749d04-21f7-4ea0-9f80-3e80f529cad6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:09.326940 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 968/1920] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:09 2022] PUT /identity/v3/projects/333b3a8c680f4e54a1f0946d4b011217/users/0fe036638eab4133b31bb30076261c17/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:09.335171 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f27f8-6be7-4573-afa7-8a2e2c59404e admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:09.337616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5b9f27f8-6be7-4573-afa7-8a2e2c59404e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:09.349435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e6749d04-21f7-4ea0-9f80-3e80f529cad6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f98ea508636e4fe49dc5fb553c10a79e', 'request_id': 'req-e6749d04-21f7-4ea0-9f80-3e80f529cad6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:09.350684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:09.366192 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e6749d04-21f7-4ea0-9f80-3e80f529cad6 admin admin] The token cache is being invalidate because project f98ea508636e4fe49dc5fb553c10a79e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:09.366492 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e6749d04-21f7-4ea0-9f80-3e80f529cad6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:09.367331 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e6749d04-21f7-4ea0-9f80-3e80f529cad6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:09.369457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 953/1921] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:09 2022] DELETE /identity/v3/projects/f98ea508636e4fe49dc5fb553c10a79e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:09.370666 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5d46d-5b78-4fc6-b856-6851a2b8bc22 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:09.371408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5d46d-5b78-4fc6-b856-6851a2b8bc22 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:09.371408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5d46d-5b78-4fc6-b856-6851a2b8bc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:09.403977 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f4b5d46d-5b78-4fc6-b856-6851a2b8bc22 None None] MFA Rules not processed for user `0fe036638eab4133b31bb30076261c17`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:09.405264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f4b5d46d-5b78-4fc6-b856-6851a2b8bc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:09.429419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f27f8-6be7-4573-afa7-8a2e2c59404e admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:09.435273 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb950b0b-a8dd-4c31-969b-ac31b24ef446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:09.436047 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb950b0b-a8dd-4c31-969b-ac31b24ef446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:09.436403 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb950b0b-a8dd-4c31-969b-ac31b24ef446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:09.436683 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb950b0b-a8dd-4c31-969b-ac31b24ef446 admin admin] PATH_INFO: `/v3/projects/ff18b88445a14bad93b5e2e9fe1e78c4` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:09.437910 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb950b0b-a8dd-4c31-969b-ac31b24ef446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff18b88445a14bad93b5e2e9fe1e78c4)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:09.439865 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb950b0b-a8dd-4c31-969b-ac31b24ef446 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:09.489444 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 954/1922] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:09.489444 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5d46d-5b78-4fc6-b856-6851a2b8bc22 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:09.489444 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f4b5d46d-5b78-4fc6-b856-6851a2b8bc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:09.502143 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-fb950b0b-a8dd-4c31-969b-ac31b24ef446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff18b88445a14bad93b5e2e9fe1e78c4', 'request_id': 'req-fb950b0b-a8dd-4c31-969b-ac31b24ef446'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:09.504934 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5d46d-5b78-4fc6-b856-6851a2b8bc22 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:09.509259 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-fb950b0b-a8dd-4c31-969b-ac31b24ef446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff18b88445a14bad93b5e2e9fe1e78c4', 'request_id': 'req-fb950b0b-a8dd-4c31-969b-ac31b24ef446'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:09.515688 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-fb950b0b-a8dd-4c31-969b-ac31b24ef446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff18b88445a14bad93b5e2e9fe1e78c4', 'request_id': 'req-fb950b0b-a8dd-4c31-969b-ac31b24ef446'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:09.529448 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-fb950b0b-a8dd-4c31-969b-ac31b24ef446 admin admin] The token cache is being invalidate because project ff18b88445a14bad93b5e2e9fe1e78c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:09.529825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-fb950b0b-a8dd-4c31-969b-ac31b24ef446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:09.530583 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-fb950b0b-a8dd-4c31-969b-ac31b24ef446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:09.532464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 969/1923] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:09 2022] DELETE /identity/v3/projects/ff18b88445a14bad93b5e2e9fe1e78c4 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:09.533071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb950b0b-a8dd-4c31-969b-ac31b24ef446 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:09.534477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-fb950b0b-a8dd-4c31-969b-ac31b24ef446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:09.574520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d5719-50af-474d-a8bc-b0c4cd4b8ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:09.575268 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d5719-50af-474d-a8bc-b0c4cd4b8ef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:09.575621 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d5719-50af-474d-a8bc-b0c4cd4b8ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:09.576039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d5719-50af-474d-a8bc-b0c4cd4b8ef2 admin admin] PATH_INFO: `/v3/users/318f2d1fb49e4b2dac41b61b1108f32b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:09.577308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5d5719-50af-474d-a8bc-b0c4cd4b8ef2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=318f2d1fb49e4b2dac41b61b1108f32b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:09.579979 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5d5719-50af-474d-a8bc-b0c4cd4b8ef2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:09.609666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb950b0b-a8dd-4c31-969b-ac31b24ef446 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:09.614871 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84458f6d-3ed6-4e30-95cd-a0ea1494881f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:09.615676 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84458f6d-3ed6-4e30-95cd-a0ea1494881f admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:09.615962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84458f6d-3ed6-4e30-95cd-a0ea1494881f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:09.616309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84458f6d-3ed6-4e30-95cd-a0ea1494881f admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:09.617015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84458f6d-3ed6-4e30-95cd-a0ea1494881f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:09.619595 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84458f6d-3ed6-4e30-95cd-a0ea1494881f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:09.639701 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3e5d5719-50af-474d-a8bc-b0c4cd4b8ef2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '318f2d1fb49e4b2dac41b61b1108f32b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:09.649081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3e5d5719-50af-474d-a8bc-b0c4cd4b8ef2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '318f2d1fb49e4b2dac41b61b1108f32b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:09.650026 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3e5d5719-50af-474d-a8bc-b0c4cd4b8ef2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '318f2d1fb49e4b2dac41b61b1108f32b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:09.650852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3e5d5719-50af-474d-a8bc-b0c4cd4b8ef2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '318f2d1fb49e4b2dac41b61b1108f32b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:09.667732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3e5d5719-50af-474d-a8bc-b0c4cd4b8ef2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '318f2d1fb49e4b2dac41b61b1108f32b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:09.678973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:09.684217 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 970/1924] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:09 2022] POST /identity/v3/projects => generated 387 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:09.684217 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84458f6d-3ed6-4e30-95cd-a0ea1494881f admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:09.686281 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-84458f6d-3ed6-4e30-95cd-a0ea1494881f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:09.689190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 955/1925] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:09 2022] DELETE /identity/v3/users/318f2d1fb49e4b2dac41b61b1108f32b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:09.690705 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d5719-50af-474d-a8bc-b0c4cd4b8ef2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:09.693596 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3e5d5719-50af-474d-a8bc-b0c4cd4b8ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:09.717345 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:09.802664 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d5719-50af-474d-a8bc-b0c4cd4b8ef2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:09.815866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6da70-b97d-45a5-9d4c-8d6b113a342d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:09.817358 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6da70-b97d-45a5-9d4c-8d6b113a342d admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:09.818226 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6da70-b97d-45a5-9d4c-8d6b113a342d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:09.820339 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6da70-b97d-45a5-9d4c-8d6b113a342d admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:09.822601 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6da70-b97d-45a5-9d4c-8d6b113a342d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:09.827813 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6da70-b97d-45a5-9d4c-8d6b113a342d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:09.855819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84458f6d-3ed6-4e30-95cd-a0ea1494881f admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:09.862434 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2608f2-7602-480e-b629-b79961afe66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:09.862434 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2608f2-7602-480e-b629-b79961afe66f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:09.863215 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2608f2-7602-480e-b629-b79961afe66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:09.863485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2608f2-7602-480e-b629-b79961afe66f admin admin] PATH_INFO: `/v3/users/0e2ade0c86a844f2a0fac1452f2c4fbf` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:09.865341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2608f2-7602-480e-b629-b79961afe66f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e2ade0c86a844f2a0fac1452f2c4fbf)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:09.873896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2608f2-7602-480e-b629-b79961afe66f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:09.886595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 956/1926] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:09 2022] POST /identity/v3/users => generated 300 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:09.893260 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6da70-b97d-45a5-9d4c-8d6b113a342d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:09.926020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6da70-b97d-45a5-9d4c-8d6b113a342d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:09.942145 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3edc8d-e746-4a43-a93e-b80d6322f666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:09.944534 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3edc8d-e746-4a43-a93e-b80d6322f666 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:09.945240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3edc8d-e746-4a43-a93e-b80d6322f666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:09.945822 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3edc8d-e746-4a43-a93e-b80d6322f666 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:09.947750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3edc8d-e746-4a43-a93e-b80d6322f666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:09.950426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3edc8d-e746-4a43-a93e-b80d6322f666 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:09.977527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 957/1927] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:09.979347 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3edc8d-e746-4a43-a93e-b80d6322f666 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:09.993971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3edc8d-e746-4a43-a93e-b80d6322f666 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:10.004473 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e3e4f-d270-4181-b70e-17850bfd3707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:10.005156 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e3e4f-d270-4181-b70e-17850bfd3707 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:10.005440 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e3e4f-d270-4181-b70e-17850bfd3707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:10.005721 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e3e4f-d270-4181-b70e-17850bfd3707 admin admin] PATH_INFO: `/v3/projects/1bf27031fcec48d69fbb0281c8d7c8c7/users/9c9a37de5c6747329be790a758cbc283/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:10.024936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-0b2608f2-7602-480e-b629-b79961afe66f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e2ade0c86a844f2a0fac1452f2c4fbf'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:10.025729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-0b2608f2-7602-480e-b629-b79961afe66f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e2ade0c86a844f2a0fac1452f2c4fbf'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:10.026472 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-0b2608f2-7602-480e-b629-b79961afe66f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e2ade0c86a844f2a0fac1452f2c4fbf'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:10.042143 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-0b2608f2-7602-480e-b629-b79961afe66f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e2ade0c86a844f2a0fac1452f2c4fbf'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:10.060607 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-0b2608f2-7602-480e-b629-b79961afe66f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e2ade0c86a844f2a0fac1452f2c4fbf'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:10.081146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e3e4f-d270-4181-b70e-17850bfd3707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bf27031fcec48d69fbb0281c8d7c8c7, user_id=9c9a37de5c6747329be790a758cbc283, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:10.088614 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 971/1928] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:09 2022] DELETE /identity/v3/users/0e2ade0c86a844f2a0fac1452f2c4fbf => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:10.093356 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e3e4f-d270-4181-b70e-17850bfd3707 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:10.124392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 958/1929] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:09 2022] PUT /identity/v3/projects/1bf27031fcec48d69fbb0281c8d7c8c7/users/9c9a37de5c6747329be790a758cbc283/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:10.128566 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2608f2-7602-480e-b629-b79961afe66f admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:10.131086 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-0b2608f2-7602-480e-b629-b79961afe66f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:10.230377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2608f2-7602-480e-b629-b79961afe66f admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:10.271736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:10.338877 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e3e4f-d270-4181-b70e-17850bfd3707 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:10.341655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1f0e3e4f-d270-4181-b70e-17850bfd3707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:10.348943 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba15675-3e16-4d4f-a2d0-3a874cdbaf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:10.349777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba15675-3e16-4d4f-a2d0-3a874cdbaf7f admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:10.350502 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba15675-3e16-4d4f-a2d0-3a874cdbaf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:10.352045 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba15675-3e16-4d4f-a2d0-3a874cdbaf7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:10.353502 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba15675-3e16-4d4f-a2d0-3a874cdbaf7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:10.358770 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba15675-3e16-4d4f-a2d0-3a874cdbaf7f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:10.374642 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e3e4f-d270-4181-b70e-17850bfd3707 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:10.386774 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 972/1930] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:10.387870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826fb5c-58b1-4386-b94f-2176e2ef030d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:10.389250 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826fb5c-58b1-4386-b94f-2176e2ef030d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:10.390390 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826fb5c-58b1-4386-b94f-2176e2ef030d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:10.390966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06b4a6-db60-4603-8662-8209de3935af None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:10.391238 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826fb5c-58b1-4386-b94f-2176e2ef030d admin admin] PATH_INFO: `/v3/projects/6f7098dc7673435f8ed3b4c2896e494e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:10.393390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06b4a6-db60-4603-8662-8209de3935af None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:10.393783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06b4a6-db60-4603-8662-8209de3935af None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:10.394456 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826fb5c-58b1-4386-b94f-2176e2ef030d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f7098dc7673435f8ed3b4c2896e494e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:10.397542 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 973/1931] 10.222.0.81 () {54 vars in 921 bytes} [Mon Sep 19 11:24:10 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:24:10.400660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826fb5c-58b1-4386-b94f-2176e2ef030d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:10.401057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06b4a6-db60-4603-8662-8209de3935af None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:10.435349 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06b4a6-db60-4603-8662-8209de3935af None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:10.450543 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c46631-60a8-48e4-8018-fb8ee3477354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:10.453505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c46631-60a8-48e4-8018-fb8ee3477354 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:10.453878 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c46631-60a8-48e4-8018-fb8ee3477354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:10.454278 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c46631-60a8-48e4-8018-fb8ee3477354 admin admin] PATH_INFO: `/v3/projects/1bf27031fcec48d69fbb0281c8d7c8c7/users/9c9a37de5c6747329be790a758cbc283/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:10.460258 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c46631-60a8-48e4-8018-fb8ee3477354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bf27031fcec48d69fbb0281c8d7c8c7, user_id=9c9a37de5c6747329be790a758cbc283, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:10.466773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c46631-60a8-48e4-8018-fb8ee3477354 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:10.492792 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 974/1932] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:10 2022] PUT /identity/v3/projects/1bf27031fcec48d69fbb0281c8d7c8c7/users/9c9a37de5c6747329be790a758cbc283/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:10.494866 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81214609-bc63-49e1-b2d4-cfbfc56777df None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:10.495158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81214609-bc63-49e1-b2d4-cfbfc56777df None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:10.495931 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81214609-bc63-49e1-b2d4-cfbfc56777df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:10.533887 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-2826fb5c-58b1-4386-b94f-2176e2ef030d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f7098dc7673435f8ed3b4c2896e494e', 'request_id': 'req-2826fb5c-58b1-4386-b94f-2176e2ef030d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:10.543246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-2826fb5c-58b1-4386-b94f-2176e2ef030d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f7098dc7673435f8ed3b4c2896e494e', 'request_id': 'req-2826fb5c-58b1-4386-b94f-2176e2ef030d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:10.552637 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-81214609-bc63-49e1-b2d4-cfbfc56777df None None] MFA Rules not processed for user `9c9a37de5c6747329be790a758cbc283`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:10.554375 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-81214609-bc63-49e1-b2d4-cfbfc56777df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:10.561434 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-2826fb5c-58b1-4386-b94f-2176e2ef030d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f7098dc7673435f8ed3b4c2896e494e', 'request_id': 'req-2826fb5c-58b1-4386-b94f-2176e2ef030d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:10.594286 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-2826fb5c-58b1-4386-b94f-2176e2ef030d admin admin] The token cache is being invalidate because project 6f7098dc7673435f8ed3b4c2896e494e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:10.596048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-2826fb5c-58b1-4386-b94f-2176e2ef030d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:10.596813 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-2826fb5c-58b1-4386-b94f-2176e2ef030d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:10.604297 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 959/1933] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:10 2022] DELETE /identity/v3/projects/6f7098dc7673435f8ed3b4c2896e494e => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:10.604297 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ae2ba-33c1-40c7-80af-37c44fa37d87 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:10.604297 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ae2ba-33c1-40c7-80af-37c44fa37d87 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:10.604297 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ae2ba-33c1-40c7-80af-37c44fa37d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:10.637541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8a1ae2ba-33c1-40c7-80af-37c44fa37d87 None None] MFA Rules not processed for user `5abd70a18ccd407bafe3b61c8870986e`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:10.638809 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8a1ae2ba-33c1-40c7-80af-37c44fa37d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:10.739722 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 975/1934] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:10.741225 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558da403-b301-4986-bfdc-2911754925d8 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:10.741579 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558da403-b301-4986-bfdc-2911754925d8 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:10.741927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558da403-b301-4986-bfdc-2911754925d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:10.769368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:10.785658 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-558da403-b301-4986-bfdc-2911754925d8 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:10.788462 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-558da403-b301-4986-bfdc-2911754925d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:10.834317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 960/1935] 10.222.0.81 () {58 vars in 1018 bytes} [Mon Sep 19 11:24:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:10.835334 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ae2ba-33c1-40c7-80af-37c44fa37d87 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:10.837675 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8a1ae2ba-33c1-40c7-80af-37c44fa37d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:10.886235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 976/1936] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:10.888810 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558da403-b301-4986-bfdc-2911754925d8 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:10.890680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-558da403-b301-4986-bfdc-2911754925d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:10.989644 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:10.998228 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ae2ba-33c1-40c7-80af-37c44fa37d87 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:11.005890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9aa70-197e-4505-80a7-cfa06397b295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:11.006641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9aa70-197e-4505-80a7-cfa06397b295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:11.007208 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9aa70-197e-4505-80a7-cfa06397b295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:11.007500 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9aa70-197e-4505-80a7-cfa06397b295 admin admin] PATH_INFO: `/v3/projects/2be63f32b9d540af8d8626aee59410da` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:11.009739 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9aa70-197e-4505-80a7-cfa06397b295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2be63f32b9d540af8d8626aee59410da)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:11.014137 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9aa70-197e-4505-80a7-cfa06397b295 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:11.043796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558da403-b301-4986-bfdc-2911754925d8 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:11.043796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0200d5e5-05cc-4e2b-ae91-c1dba8be6e3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:11.043796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200d5e5-05cc-4e2b-ae91-c1dba8be6e3d service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:11.043796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200d5e5-05cc-4e2b-ae91-c1dba8be6e3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:11.044319 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200d5e5-05cc-4e2b-ae91-c1dba8be6e3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:11.066013 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0200d5e5-05cc-4e2b-ae91-c1dba8be6e3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:11.069834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0200d5e5-05cc-4e2b-ae91-c1dba8be6e3d service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:11.100891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-27d9aa70-197e-4505-80a7-cfa06397b295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2be63f32b9d540af8d8626aee59410da', 'request_id': 'req-27d9aa70-197e-4505-80a7-cfa06397b295'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:11.110019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-27d9aa70-197e-4505-80a7-cfa06397b295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2be63f32b9d540af8d8626aee59410da', 'request_id': 'req-27d9aa70-197e-4505-80a7-cfa06397b295'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:11.112522 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 977/1937] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:10 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:11.113964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452dbdb9-2845-41ff-93d8-1b083b8ed7a8 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:11.114248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452dbdb9-2845-41ff-93d8-1b083b8ed7a8 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:11.114526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452dbdb9-2845-41ff-93d8-1b083b8ed7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:11.116574 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-27d9aa70-197e-4505-80a7-cfa06397b295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2be63f32b9d540af8d8626aee59410da', 'request_id': 'req-27d9aa70-197e-4505-80a7-cfa06397b295'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:11.142268 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-27d9aa70-197e-4505-80a7-cfa06397b295 admin admin] The token cache is being invalidate because project 2be63f32b9d540af8d8626aee59410da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:11.142591 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-27d9aa70-197e-4505-80a7-cfa06397b295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:11.143880 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-27d9aa70-197e-4505-80a7-cfa06397b295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:11.145916 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 961/1938] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:10 2022] DELETE /identity/v3/projects/2be63f32b9d540af8d8626aee59410da => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:11.147909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9aa70-197e-4505-80a7-cfa06397b295 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:11.152906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-27d9aa70-197e-4505-80a7-cfa06397b295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:11.160666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-452dbdb9-2845-41ff-93d8-1b083b8ed7a8 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:11.162368 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-452dbdb9-2845-41ff-93d8-1b083b8ed7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:11.270819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 978/1939] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:11.276733 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3cbb86-0b5a-4ad5-bafe-a0dde2e09d4d None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:11.277176 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3cbb86-0b5a-4ad5-bafe-a0dde2e09d4d None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:11.277608 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3cbb86-0b5a-4ad5-bafe-a0dde2e09d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:11.284454 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:11.312435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9aa70-197e-4505-80a7-cfa06397b295 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:11.314551 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-da3cbb86-0b5a-4ad5-bafe-a0dde2e09d4d None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:11.317390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-da3cbb86-0b5a-4ad5-bafe-a0dde2e09d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:11.320168 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd00c65-0fcd-48dd-8003-330a6b9ee4ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:11.320901 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd00c65-0fcd-48dd-8003-330a6b9ee4ad service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:11.321944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd00c65-0fcd-48dd-8003-330a6b9ee4ad service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:11.322315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd00c65-0fcd-48dd-8003-330a6b9ee4ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:11.324824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4fd00c65-0fcd-48dd-8003-330a6b9ee4ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:11.332361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 979/1940] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:11.334166 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3cbb86-0b5a-4ad5-bafe-a0dde2e09d4d None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:11.354945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3cbb86-0b5a-4ad5-bafe-a0dde2e09d4d None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:11.355315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd00c65-0fcd-48dd-8003-330a6b9ee4ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:11.358674 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd00c65-0fcd-48dd-8003-330a6b9ee4ad service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:11.365212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6deb7-0cdc-4fd1-bb7a-6c7a71e4ca46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:11.367074 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6deb7-0cdc-4fd1-bb7a-6c7a71e4ca46 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:11.367830 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6deb7-0cdc-4fd1-bb7a-6c7a71e4ca46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:11.369083 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6deb7-0cdc-4fd1-bb7a-6c7a71e4ca46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:11.370431 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6deb7-0cdc-4fd1-bb7a-6c7a71e4ca46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:11.371226 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6deb7-0cdc-4fd1-bb7a-6c7a71e4ca46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:11.376278 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6deb7-0cdc-4fd1-bb7a-6c7a71e4ca46 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:11.396197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 962/1941] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:11.398605 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7738e00-b99c-4224-9316-67c0465e71b0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:11.399964 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7738e00-b99c-4224-9316-67c0465e71b0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:11.400447 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7738e00-b99c-4224-9316-67c0465e71b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:11.412848 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 980/1942] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:11.422407 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6deb7-0cdc-4fd1-bb7a-6c7a71e4ca46 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:11.441149 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c7738e00-b99c-4224-9316-67c0465e71b0 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:11.444077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c7738e00-b99c-4224-9316-67c0465e71b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:11.447232 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6deb7-0cdc-4fd1-bb7a-6c7a71e4ca46 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:11.460127 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fa766-7c2b-471e-92a1-dec92513ae51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:11.460750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fa766-7c2b-471e-92a1-dec92513ae51 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:11.461031 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fa766-7c2b-471e-92a1-dec92513ae51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:11.461307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fa766-7c2b-471e-92a1-dec92513ae51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:11.461456 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 963/1943] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:11.467116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fa766-7c2b-471e-92a1-dec92513ae51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:11.467116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fa766-7c2b-471e-92a1-dec92513ae51 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:11.469435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7738e00-b99c-4224-9316-67c0465e71b0 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:11.484192 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7738e00-b99c-4224-9316-67c0465e71b0 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:11.506602 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc4b80-6779-418c-a320-e751dd7262a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:11.507392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc4b80-6779-418c-a320-e751dd7262a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:11.507868 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc4b80-6779-418c-a320-e751dd7262a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:11.508299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc4b80-6779-418c-a320-e751dd7262a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:11.509053 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 981/1944] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:11 2022] POST /identity/v3/projects => generated 379 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:11.509546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afc4b80-6779-418c-a320-e751dd7262a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:11.510165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afc4b80-6779-418c-a320-e751dd7262a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:11.511938 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afc4b80-6779-418c-a320-e751dd7262a3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:11.513045 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fa766-7c2b-471e-92a1-dec92513ae51 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:11.528898 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 964/1945] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:11.530227 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc4b80-6779-418c-a320-e751dd7262a3 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:11.543659 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc4b80-6779-418c-a320-e751dd7262a3 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:11.597961 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fa766-7c2b-471e-92a1-dec92513ae51 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:11.604795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e4542-b1ba-4095-8230-a12e0c4d9edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:11.605416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e4542-b1ba-4095-8230-a12e0c4d9edd admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:11.605962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e4542-b1ba-4095-8230-a12e0c4d9edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:11.605962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e4542-b1ba-4095-8230-a12e0c4d9edd admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:11.606593 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e4542-b1ba-4095-8230-a12e0c4d9edd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:11.608541 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e4542-b1ba-4095-8230-a12e0c4d9edd admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:11.632209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632db084-9466-4d42-8d57-6aceeed438eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:11.632958 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632db084-9466-4d42-8d57-6aceeed438eb admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:11.633240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632db084-9466-4d42-8d57-6aceeed438eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:11.634883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632db084-9466-4d42-8d57-6aceeed438eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:11.635378 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 982/1946] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:11 2022] POST /identity/v3/users => generated 290 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:24:11.636496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632db084-9466-4d42-8d57-6aceeed438eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:11.638846 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e4542-b1ba-4095-8230-a12e0c4d9edd admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:11.641026 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632db084-9466-4d42-8d57-6aceeed438eb admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:11.656044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e4542-b1ba-4095-8230-a12e0c4d9edd admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:11.662402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c0511-25f6-483b-ad06-1d7b111215a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:11.663122 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c0511-25f6-483b-ad06-1d7b111215a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:11.663464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c0511-25f6-483b-ad06-1d7b111215a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:11.663703 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c0511-25f6-483b-ad06-1d7b111215a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:11.664515 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c0511-25f6-483b-ad06-1d7b111215a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:11.666184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c0511-25f6-483b-ad06-1d7b111215a4 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:11.683529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 983/1947] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:11.683933 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 965/1948] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:11 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:11.684809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c0511-25f6-483b-ad06-1d7b111215a4 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:11.685034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632db084-9466-4d42-8d57-6aceeed438eb admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:11.701759 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c0511-25f6-483b-ad06-1d7b111215a4 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:11.734745 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:11.767097 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632db084-9466-4d42-8d57-6aceeed438eb admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:11.773084 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76dd255-f0f6-4698-9f65-d0a0d7307267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:11.773699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dd255-f0f6-4698-9f65-d0a0d7307267 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:11.773980 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dd255-f0f6-4698-9f65-d0a0d7307267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:11.774283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dd255-f0f6-4698-9f65-d0a0d7307267 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:11.774957 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76dd255-f0f6-4698-9f65-d0a0d7307267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:11.776972 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76dd255-f0f6-4698-9f65-d0a0d7307267 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:11.809467 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 966/1949] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:11 2022] POST /identity/v3/users => generated 290 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:11.809638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4229067-a55e-4ff4-8bcf-fc766d145b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:11.809638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4229067-a55e-4ff4-8bcf-fc766d145b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:11.809638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4229067-a55e-4ff4-8bcf-fc766d145b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:11.809638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4229067-a55e-4ff4-8bcf-fc766d145b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:11.809994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4229067-a55e-4ff4-8bcf-fc766d145b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:11.811166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76dd255-f0f6-4698-9f65-d0a0d7307267 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:11.811278 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4229067-a55e-4ff4-8bcf-fc766d145b0d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:11.822625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76dd255-f0f6-4698-9f65-d0a0d7307267 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:11.826043 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 984/1950] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:11.826918 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4229067-a55e-4ff4-8bcf-fc766d145b0d admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:11.827801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fc091-2060-418c-87a6-51bae932b989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:11.828486 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fc091-2060-418c-87a6-51bae932b989 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:11.828855 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fc091-2060-418c-87a6-51bae932b989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:11.829309 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fc091-2060-418c-87a6-51bae932b989 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:11.830019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70fc091-2060-418c-87a6-51bae932b989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:11.831718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70fc091-2060-418c-87a6-51bae932b989 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:11.839965 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4229067-a55e-4ff4-8bcf-fc766d145b0d admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:11.842190 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47506d3c-4bc7-457d-999f-935ada434eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:11.842711 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 967/1951] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:24:11.842902 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47506d3c-4bc7-457d-999f-935ada434eb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:11.843249 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47506d3c-4bc7-457d-999f-935ada434eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:11.843533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47506d3c-4bc7-457d-999f-935ada434eb6 admin admin] PATH_INFO: `/v3/projects/0c83eef359544b99bb4a71b47790ede8/users/9e95d3c6e5fd43e4baa5bc577ffaf721/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:11.846414 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fc091-2060-418c-87a6-51bae932b989 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:11.857072 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fc091-2060-418c-87a6-51bae932b989 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:11.861852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bba9bf-1f5c-4023-9f73-f0ef9bd6e8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:11.862450 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bba9bf-1f5c-4023-9f73-f0ef9bd6e8ef admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:11.862551 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47506d3c-4bc7-457d-999f-935ada434eb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c83eef359544b99bb4a71b47790ede8, user_id=9e95d3c6e5fd43e4baa5bc577ffaf721, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:11.862730 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bba9bf-1f5c-4023-9f73-f0ef9bd6e8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:11.863013 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bba9bf-1f5c-4023-9f73-f0ef9bd6e8ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:11.863637 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bba9bf-1f5c-4023-9f73-f0ef9bd6e8ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:11.865278 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bba9bf-1f5c-4023-9f73-f0ef9bd6e8ef admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:11.865593 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47506d3c-4bc7-457d-999f-935ada434eb6 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:11.878075 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 968/1952] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:24:11.881952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 985/1953] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:11 2022] PUT /identity/v3/projects/0c83eef359544b99bb4a71b47790ede8/users/9e95d3c6e5fd43e4baa5bc577ffaf721/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:11.882274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bba9bf-1f5c-4023-9f73-f0ef9bd6e8ef admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:11.891692 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fff92e-7716-4a71-859d-7b92ae3ae589 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:11.891991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fff92e-7716-4a71-859d-7b92ae3ae589 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:11.892264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fff92e-7716-4a71-859d-7b92ae3ae589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:11.893061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bba9bf-1f5c-4023-9f73-f0ef9bd6e8ef admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:11.906580 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:11.928457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-d6fff92e-7716-4a71-859d-7b92ae3ae589 None None] MFA Rules not processed for user `9e95d3c6e5fd43e4baa5bc577ffaf721`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:11.929749 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-d6fff92e-7716-4a71-859d-7b92ae3ae589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:11.956079 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf29d49-58cb-4200-aa61-6fbe0e872912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:11.957249 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf29d49-58cb-4200-aa61-6fbe0e872912 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:11.957684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf29d49-58cb-4200-aa61-6fbe0e872912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:11.957995 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf29d49-58cb-4200-aa61-6fbe0e872912 admin admin] PATH_INFO: `/v3/projects/a72e725b681e4561aa7764b426c0910d/users/9d2a9b0e225147d9b9558d136763fe00/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:11.973131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf29d49-58cb-4200-aa61-6fbe0e872912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a72e725b681e4561aa7764b426c0910d, user_id=9d2a9b0e225147d9b9558d136763fe00, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:11.975935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf29d49-58cb-4200-aa61-6fbe0e872912 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:11.990096 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 969/1954] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:11 2022] PUT /identity/v3/projects/a72e725b681e4561aa7764b426c0910d/users/9d2a9b0e225147d9b9558d136763fe00/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:11.999124 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf29d49-58cb-4200-aa61-6fbe0e872912 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:11.999124 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4cf29d49-58cb-4200-aa61-6fbe0e872912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:11.999390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 986/1955] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:12.000184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fff92e-7716-4a71-859d-7b92ae3ae589 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:12.011089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fff92e-7716-4a71-859d-7b92ae3ae589 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:12.053327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf29d49-58cb-4200-aa61-6fbe0e872912 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:12.057696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ac770-85d5-40a7-8d1d-37a781db2687 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:12.058305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ac770-85d5-40a7-8d1d-37a781db2687 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:12.058583 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ac770-85d5-40a7-8d1d-37a781db2687 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:12.058857 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ac770-85d5-40a7-8d1d-37a781db2687 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:12.060653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-089ac770-85d5-40a7-8d1d-37a781db2687 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:12.081384 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3c278-f93c-4890-b678-c685ce484a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:12.082054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3c278-f93c-4890-b678-c685ce484a71 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:12.082424 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3c278-f93c-4890-b678-c685ce484a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:12.082609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3c278-f93c-4890-b678-c685ce484a71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:12.083250 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3c278-f93c-4890-b678-c685ce484a71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:12.085463 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3c278-f93c-4890-b678-c685ce484a71 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:12.143097 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 987/1956] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:11 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:12.147963 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ac770-85d5-40a7-8d1d-37a781db2687 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:12.148524 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6922b-51a5-4921-8088-121c667d021d None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:12.149334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6922b-51a5-4921-8088-121c667d021d None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:12.149334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6922b-51a5-4921-8088-121c667d021d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:12.149888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ac770-85d5-40a7-8d1d-37a781db2687 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:12.174776 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 970/1957] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:12.175635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ac770-85d5-40a7-8d1d-37a781db2687 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:12.187411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ac770-85d5-40a7-8d1d-37a781db2687 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:12.193748 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:12.206127 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-31f6922b-51a5-4921-8088-121c667d021d None None] MFA Rules not processed for user `9d2a9b0e225147d9b9558d136763fe00`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:12.208189 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-31f6922b-51a5-4921-8088-121c667d021d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:12.261805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:12.276627 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691485d-86e1-4be4-bff2-d32a03245262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:12.277356 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691485d-86e1-4be4-bff2-d32a03245262 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:12.277547 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691485d-86e1-4be4-bff2-d32a03245262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:12.277830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691485d-86e1-4be4-bff2-d32a03245262 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:12.278687 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691485d-86e1-4be4-bff2-d32a03245262 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:12.280808 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691485d-86e1-4be4-bff2-d32a03245262 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:12.305064 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 988/1958] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:12.308396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 971/1959] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:12 2022] POST /identity/v3/users => generated 297 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:12.310047 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6922b-51a5-4921-8088-121c667d021d None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:12.312324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691485d-86e1-4be4-bff2-d32a03245262 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:12.331981 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691485d-86e1-4be4-bff2-d32a03245262 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:12.332147 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6922b-51a5-4921-8088-121c667d021d None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:12.340976 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a02451-cd53-48ea-805e-bbf485c4b819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:12.342487 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a02451-cd53-48ea-805e-bbf485c4b819 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:12.342772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a02451-cd53-48ea-805e-bbf485c4b819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:12.343103 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a02451-cd53-48ea-805e-bbf485c4b819 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:12.343864 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a02451-cd53-48ea-805e-bbf485c4b819 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:12.345908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a02451-cd53-48ea-805e-bbf485c4b819 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:12.365917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c4271-6ac8-416d-b0f9-e06b6c343d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:12.366707 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c4271-6ac8-416d-b0f9-e06b6c343d70 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:12.366994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c4271-6ac8-416d-b0f9-e06b6c343d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:12.367406 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c4271-6ac8-416d-b0f9-e06b6c343d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:12.370569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c4271-6ac8-416d-b0f9-e06b6c343d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:12.371043 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c4271-6ac8-416d-b0f9-e06b6c343d70 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:12.389701 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 972/1960] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:12.406340 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c4271-6ac8-416d-b0f9-e06b6c343d70 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:12.406678 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 989/1961] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:12 2022] POST /identity/v3/projects => generated 377 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:24:12.406756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c4271-6ac8-416d-b0f9-e06b6c343d70 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:12.406825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a02451-cd53-48ea-805e-bbf485c4b819 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:12.491720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c59c26-0562-474e-9aa3-ab98e455d574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:12.492734 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c59c26-0562-474e-9aa3-ab98e455d574 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:12.494163 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c59c26-0562-474e-9aa3-ab98e455d574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:12.494540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c59c26-0562-474e-9aa3-ab98e455d574 admin admin] PATH_INFO: `/v3/projects/99be19f7a21142108d05d355b21d6186/users/09ceeef2a0844fe2bef9bfb9b9c09deb/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:12.525685 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a02451-cd53-48ea-805e-bbf485c4b819 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:12.526362 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c59c26-0562-474e-9aa3-ab98e455d574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99be19f7a21142108d05d355b21d6186, user_id=09ceeef2a0844fe2bef9bfb9b9c09deb, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:12.529135 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c59c26-0562-474e-9aa3-ab98e455d574 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:12.534970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f9559-d30f-4af2-83e3-cf855482142c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:12.535810 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f9559-d30f-4af2-83e3-cf855482142c admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:12.536167 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f9559-d30f-4af2-83e3-cf855482142c admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:12.536560 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f9559-d30f-4af2-83e3-cf855482142c admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:12.537415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f9559-d30f-4af2-83e3-cf855482142c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:12.539633 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f9559-d30f-4af2-83e3-cf855482142c admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:12.549882 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 973/1962] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:12 2022] PUT /identity/v3/projects/99be19f7a21142108d05d355b21d6186/users/09ceeef2a0844fe2bef9bfb9b9c09deb/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:12.556661 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c59c26-0562-474e-9aa3-ab98e455d574 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:12.574386 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 990/1963] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:12 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:24:12.578266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f9559-d30f-4af2-83e3-cf855482142c admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:12.589205 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f9559-d30f-4af2-83e3-cf855482142c admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:12.650921 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c59c26-0562-474e-9aa3-ab98e455d574 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:12.659445 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f604895-d3a3-4f7d-82d5-abf24fa6160a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:12.660857 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f604895-d3a3-4f7d-82d5-abf24fa6160a admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:12.661141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f604895-d3a3-4f7d-82d5-abf24fa6160a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:12.661446 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f604895-d3a3-4f7d-82d5-abf24fa6160a admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:12.662109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f604895-d3a3-4f7d-82d5-abf24fa6160a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:12.664257 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f604895-d3a3-4f7d-82d5-abf24fa6160a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:12.679950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1c258-6269-472e-8728-ce5f33f347cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:12.680471 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 974/1964] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:12.680639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1c258-6269-472e-8728-ce5f33f347cd admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:12.680929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1c258-6269-472e-8728-ce5f33f347cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:12.681186 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1c258-6269-472e-8728-ce5f33f347cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:12.681827 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1c258-6269-472e-8728-ce5f33f347cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:12.683490 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1c258-6269-472e-8728-ce5f33f347cd admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:12.684338 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f604895-d3a3-4f7d-82d5-abf24fa6160a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:12.693862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 991/1965] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:12.694083 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f604895-d3a3-4f7d-82d5-abf24fa6160a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:12.698706 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1c258-6269-472e-8728-ce5f33f347cd admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:12.699716 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2664bd8-2b83-4975-8456-254aaacf0214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:12.700398 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2664bd8-2b83-4975-8456-254aaacf0214 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:12.700679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2664bd8-2b83-4975-8456-254aaacf0214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:12.700953 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2664bd8-2b83-4975-8456-254aaacf0214 admin admin] PATH_INFO: `/v3/projects/99be19f7a21142108d05d355b21d6186/users/09ceeef2a0844fe2bef9bfb9b9c09deb/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:12.702866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2664bd8-2b83-4975-8456-254aaacf0214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99be19f7a21142108d05d355b21d6186, user_id=09ceeef2a0844fe2bef9bfb9b9c09deb, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:12.703754 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:12.706178 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2664bd8-2b83-4975-8456-254aaacf0214 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:12.710705 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1c258-6269-472e-8728-ce5f33f347cd admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:12.717747 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 975/1966] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:12 2022] PUT /identity/v3/projects/99be19f7a21142108d05d355b21d6186/users/09ceeef2a0844fe2bef9bfb9b9c09deb/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:12.728429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52b24e-64ca-4919-adca-d3ba9e560a79 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:12.729117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52b24e-64ca-4919-adca-d3ba9e560a79 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:12.729117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52b24e-64ca-4919-adca-d3ba9e560a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:12.763249 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-ac52b24e-64ca-4919-adca-d3ba9e560a79 None None] MFA Rules not processed for user `09ceeef2a0844fe2bef9bfb9b9c09deb`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:12.764768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ac52b24e-64ca-4919-adca-d3ba9e560a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:12.778055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11ac96-1c6f-491f-ae7c-f34a322c22d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:12.778666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11ac96-1c6f-491f-ae7c-f34a322c22d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:12.778951 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11ac96-1c6f-491f-ae7c-f34a322c22d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:12.779229 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11ac96-1c6f-491f-ae7c-f34a322c22d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:12.779906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e11ac96-1c6f-491f-ae7c-f34a322c22d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:12.781490 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e11ac96-1c6f-491f-ae7c-f34a322c22d7 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:12.794005 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 992/1967] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:24:12.798327 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11ac96-1c6f-491f-ae7c-f34a322c22d7 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:12.802943 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:12.810109 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11ac96-1c6f-491f-ae7c-f34a322c22d7 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:12.815120 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070cce1-4d4e-423b-8c80-51778e552292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:12.815804 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070cce1-4d4e-423b-8c80-51778e552292 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:12.816082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070cce1-4d4e-423b-8c80-51778e552292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:12.816357 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070cce1-4d4e-423b-8c80-51778e552292 admin admin] PATH_INFO: `/v3/projects/97e8a82e81ec4fc5bd40354c4bf02b39/users/09f5fdd3849449c192e16d8091348f16/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:12.832803 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070cce1-4d4e-423b-8c80-51778e552292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97e8a82e81ec4fc5bd40354c4bf02b39, user_id=09f5fdd3849449c192e16d8091348f16, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:12.835756 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070cce1-4d4e-423b-8c80-51778e552292 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:12.863168 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 976/1969] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:24:12.864325 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 993/1969] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:12 2022] PUT /identity/v3/projects/97e8a82e81ec4fc5bd40354c4bf02b39/users/09f5fdd3849449c192e16d8091348f16/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:12.865182 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52b24e-64ca-4919-adca-d3ba9e560a79 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:12.866668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d70a7-87cf-4412-89b5-30f21749845a None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:12.866960 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d70a7-87cf-4412-89b5-30f21749845a None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:12.870281 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d70a7-87cf-4412-89b5-30f21749845a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:12.900056 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-ea1d70a7-87cf-4412-89b5-30f21749845a None None] MFA Rules not processed for user `09f5fdd3849449c192e16d8091348f16`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:12.901355 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ea1d70a7-87cf-4412-89b5-30f21749845a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:12.931027 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52b24e-64ca-4919-adca-d3ba9e560a79 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:12.978971 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 994/1970] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:12.983944 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d70a7-87cf-4412-89b5-30f21749845a None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:12.985803 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea51edb-17ef-400e-8142-567bdeb4687d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:12.986414 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea51edb-17ef-400e-8142-567bdeb4687d service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:12.986710 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea51edb-17ef-400e-8142-567bdeb4687d service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:12.986962 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea51edb-17ef-400e-8142-567bdeb4687d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:13.050395 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d70a7-87cf-4412-89b5-30f21749845a None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:13.053057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea51edb-17ef-400e-8142-567bdeb4687d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:13.054571 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea51edb-17ef-400e-8142-567bdeb4687d service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:13.055047 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77676dac-1f4a-44a6-84d9-547c5019d5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:13.056156 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77676dac-1f4a-44a6-84d9-547c5019d5bb admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:13.056435 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77676dac-1f4a-44a6-84d9-547c5019d5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:13.056702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77676dac-1f4a-44a6-84d9-547c5019d5bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:13.057334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77676dac-1f4a-44a6-84d9-547c5019d5bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:13.059216 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77676dac-1f4a-44a6-84d9-547c5019d5bb admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:13.065294 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 977/1971] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:12 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:24:13.069476 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:13.090036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 995/1972] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:12 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:24:13.094116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea51edb-17ef-400e-8142-567bdeb4687d service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:13.109683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea51edb-17ef-400e-8142-567bdeb4687d service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:13.135218 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:13.169404 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71426ad-d97d-4169-81c6-a844a1ad2552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:13.170329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71426ad-d97d-4169-81c6-a844a1ad2552 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:13.170329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71426ad-d97d-4169-81c6-a844a1ad2552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:13.170536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71426ad-d97d-4169-81c6-a844a1ad2552 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:13.171168 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71426ad-d97d-4169-81c6-a844a1ad2552 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:13.173267 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71426ad-d97d-4169-81c6-a844a1ad2552 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:13.207117 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77676dac-1f4a-44a6-84d9-547c5019d5bb admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:13.207243 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 978/1973] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:13 2022] POST /identity/v3/users => generated 295 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:13.219122 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77676dac-1f4a-44a6-84d9-547c5019d5bb admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:13.224441 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc7bf8-dfb9-4cb5-9a3f-7e479804c00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:13.225054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc7bf8-dfb9-4cb5-9a3f-7e479804c00b admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:13.225330 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc7bf8-dfb9-4cb5-9a3f-7e479804c00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:13.225600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc7bf8-dfb9-4cb5-9a3f-7e479804c00b admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:13.226247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc7bf8-dfb9-4cb5-9a3f-7e479804c00b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:13.228814 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc7bf8-dfb9-4cb5-9a3f-7e479804c00b admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:13.239827 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 996/1974] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:13.244184 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71426ad-d97d-4169-81c6-a844a1ad2552 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:13.260936 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71426ad-d97d-4169-81c6-a844a1ad2552 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:13.276931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a62ae-7b11-4591-aa1d-9acd4efc9136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:13.278802 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a62ae-7b11-4591-aa1d-9acd4efc9136 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:13.278802 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a62ae-7b11-4591-aa1d-9acd4efc9136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:13.278802 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a62ae-7b11-4591-aa1d-9acd4efc9136 admin admin] PATH_INFO: `/v3/projects/cea2ee65ace7429490a6dbb458953fc9/users/c748bbbe47564bfa9d67d587dd83b704/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:13.295023 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a62ae-7b11-4591-aa1d-9acd4efc9136 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea2ee65ace7429490a6dbb458953fc9, user_id=c748bbbe47564bfa9d67d587dd83b704, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:13.299889 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a62ae-7b11-4591-aa1d-9acd4efc9136 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:13.316136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 979/1975] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:13 2022] PUT /identity/v3/projects/cea2ee65ace7429490a6dbb458953fc9/users/c748bbbe47564bfa9d67d587dd83b704/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:13.320032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc7bf8-dfb9-4cb5-9a3f-7e479804c00b admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:13.332671 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc7bf8-dfb9-4cb5-9a3f-7e479804c00b admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:13.413589 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e55cde-45f6-4b6d-bf66-3a0a3f50565b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:13.414681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e55cde-45f6-4b6d-bf66-3a0a3f50565b admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:13.415443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e55cde-45f6-4b6d-bf66-3a0a3f50565b admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:13.415971 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e55cde-45f6-4b6d-bf66-3a0a3f50565b admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:13.416820 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e55cde-45f6-4b6d-bf66-3a0a3f50565b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:13.419043 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e55cde-45f6-4b6d-bf66-3a0a3f50565b admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:13.437417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 997/1976] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:13.440407 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a62ae-7b11-4591-aa1d-9acd4efc9136 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:13.455325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a62ae-7b11-4591-aa1d-9acd4efc9136 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:13.461497 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e8d79-7513-409d-8f9f-57a882d6c1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:13.463064 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e8d79-7513-409d-8f9f-57a882d6c1a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:13.463661 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e8d79-7513-409d-8f9f-57a882d6c1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:13.465076 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e8d79-7513-409d-8f9f-57a882d6c1a8 admin admin] PATH_INFO: `/v3/projects/cea2ee65ace7429490a6dbb458953fc9/users/c748bbbe47564bfa9d67d587dd83b704/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:13.468987 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e8d79-7513-409d-8f9f-57a882d6c1a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea2ee65ace7429490a6dbb458953fc9, user_id=c748bbbe47564bfa9d67d587dd83b704, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:13.472320 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e8d79-7513-409d-8f9f-57a882d6c1a8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:13.490971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 980/1977] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:13 2022] PUT /identity/v3/projects/cea2ee65ace7429490a6dbb458953fc9/users/c748bbbe47564bfa9d67d587dd83b704/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:13.502017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdc257-f0c1-431d-aa15-39189c2c4ed9 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:13.502388 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdc257-f0c1-431d-aa15-39189c2c4ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:13.502668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdc257-f0c1-431d-aa15-39189c2c4ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:13.537052 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-4dbdc257-f0c1-431d-aa15-39189c2c4ed9 None None] MFA Rules not processed for user `c748bbbe47564bfa9d67d587dd83b704`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:13.538517 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-4dbdc257-f0c1-431d-aa15-39189c2c4ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:13.547757 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e8d79-7513-409d-8f9f-57a882d6c1a8 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:13.623727 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 998/1978] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:13.635272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e8d79-7513-409d-8f9f-57a882d6c1a8 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:13.640188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b0d7e-84ba-4c7f-bc04-ad22b767459c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:13.640955 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b0d7e-84ba-4c7f-bc04-ad22b767459c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:13.641349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b0d7e-84ba-4c7f-bc04-ad22b767459c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:13.641720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b0d7e-84ba-4c7f-bc04-ad22b767459c admin admin] PATH_INFO: `/v3/users/9e95d3c6e5fd43e4baa5bc577ffaf721` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:13.642963 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b0d7e-84ba-4c7f-bc04-ad22b767459c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e95d3c6e5fd43e4baa5bc577ffaf721)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:13.643922 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbdc257-f0c1-431d-aa15-39189c2c4ed9 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:13.645147 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b0d7e-84ba-4c7f-bc04-ad22b767459c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:13.721412 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbdc257-f0c1-431d-aa15-39189c2c4ed9 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:13.725559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ad67d-2cae-4977-b104-7380e3d7ac5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:13.726489 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ad67d-2cae-4977-b104-7380e3d7ac5e service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:13.726785 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ad67d-2cae-4977-b104-7380e3d7ac5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:13.727034 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ad67d-2cae-4977-b104-7380e3d7ac5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:13.733060 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-485b0d7e-84ba-4c7f-bc04-ad22b767459c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e95d3c6e5fd43e4baa5bc577ffaf721'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:13.738694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ad67d-2cae-4977-b104-7380e3d7ac5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:13.740318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ad67d-2cae-4977-b104-7380e3d7ac5e service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:13.741871 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-485b0d7e-84ba-4c7f-bc04-ad22b767459c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e95d3c6e5fd43e4baa5bc577ffaf721'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:13.742676 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-485b0d7e-84ba-4c7f-bc04-ad22b767459c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e95d3c6e5fd43e4baa5bc577ffaf721'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:13.743335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-485b0d7e-84ba-4c7f-bc04-ad22b767459c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e95d3c6e5fd43e4baa5bc577ffaf721'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:13.757185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 999/1979] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:24:13.772521 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-485b0d7e-84ba-4c7f-bc04-ad22b767459c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e95d3c6e5fd43e4baa5bc577ffaf721'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:13.792008 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 981/1980] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:13 2022] DELETE /identity/v3/users/9e95d3c6e5fd43e4baa5bc577ffaf721 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:13.795171 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0c883-4033-46ae-8b66-c29bff6a40ae None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:13.795636 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0c883-4033-46ae-8b66-c29bff6a40ae None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:13.796027 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0c883-4033-46ae-8b66-c29bff6a40ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:13.804336 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b0d7e-84ba-4c7f-bc04-ad22b767459c admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:13.806449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-485b0d7e-84ba-4c7f-bc04-ad22b767459c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:13.808295 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:13.815324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:13.828836 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-c3b0c883-4033-46ae-8b66-c29bff6a40ae None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:13.830071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c3b0c883-4033-46ae-8b66-c29bff6a40ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:13.922582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1000/1981] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:13.929557 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0c883-4033-46ae-8b66-c29bff6a40ae None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:13.931714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c3b0c883-4033-46ae-8b66-c29bff6a40ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:13.936519 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b0d7e-84ba-4c7f-bc04-ad22b767459c admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:13.952662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff1959-f536-4e3d-b5a6-e73db515adee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:13.954296 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff1959-f536-4e3d-b5a6-e73db515adee service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:13.954665 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff1959-f536-4e3d-b5a6-e73db515adee service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:13.955010 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff1959-f536-4e3d-b5a6-e73db515adee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:13.957371 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4aff1959-f536-4e3d-b5a6-e73db515adee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:14.019218 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0c883-4033-46ae-8b66-c29bff6a40ae None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:14.026819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0a85f-a5fa-42b8-853b-1995c082cab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:14.027451 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0a85f-a5fa-42b8-853b-1995c082cab6 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:14.027796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0a85f-a5fa-42b8-853b-1995c082cab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:14.028183 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0a85f-a5fa-42b8-853b-1995c082cab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:14.042972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0a85f-a5fa-42b8-853b-1995c082cab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:14.044442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0a85f-a5fa-42b8-853b-1995c082cab6 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:14.068651 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1001/1982] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:14.168346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff1959-f536-4e3d-b5a6-e73db515adee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:14.173961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff1959-f536-4e3d-b5a6-e73db515adee service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:14.186408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 982/1983] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:24:14.423991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0a85f-a5fa-42b8-853b-1995c082cab6 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:14.427212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-23a0a85f-a5fa-42b8-853b-1995c082cab6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:14.448258 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0a85f-a5fa-42b8-853b-1995c082cab6 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:14.454788 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7f4c6-699a-4a55-b8b0-9d824e18bcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:14.456220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7f4c6-699a-4a55-b8b0-9d824e18bcde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:14.456666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7f4c6-699a-4a55-b8b0-9d824e18bcde admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:14.457097 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7f4c6-699a-4a55-b8b0-9d824e18bcde admin admin] PATH_INFO: `/v3/projects/0c83eef359544b99bb4a71b47790ede8` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:14.459146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7f4c6-699a-4a55-b8b0-9d824e18bcde admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c83eef359544b99bb4a71b47790ede8)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:14.461326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7f4c6-699a-4a55-b8b0-9d824e18bcde admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:14.548477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-36f7f4c6-699a-4a55-b8b0-9d824e18bcde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c83eef359544b99bb4a71b47790ede8', 'request_id': 'req-36f7f4c6-699a-4a55-b8b0-9d824e18bcde'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:14.558734 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-36f7f4c6-699a-4a55-b8b0-9d824e18bcde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c83eef359544b99bb4a71b47790ede8', 'request_id': 'req-36f7f4c6-699a-4a55-b8b0-9d824e18bcde'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:14.565871 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-36f7f4c6-699a-4a55-b8b0-9d824e18bcde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c83eef359544b99bb4a71b47790ede8', 'request_id': 'req-36f7f4c6-699a-4a55-b8b0-9d824e18bcde'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:14.583204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-36f7f4c6-699a-4a55-b8b0-9d824e18bcde admin admin] The token cache is being invalidate because project 0c83eef359544b99bb4a71b47790ede8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:14.583652 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-36f7f4c6-699a-4a55-b8b0-9d824e18bcde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:14.584668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-36f7f4c6-699a-4a55-b8b0-9d824e18bcde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:14.586312 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1002/1984] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:14 2022] DELETE /identity/v3/projects/0c83eef359544b99bb4a71b47790ede8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:14.591328 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff1959-f536-4e3d-b5a6-e73db515adee service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:14.636844 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff1959-f536-4e3d-b5a6-e73db515adee service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:14.810732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecfd3a-f1a8-45d1-8248-767b91a198fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:14.811590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecfd3a-f1a8-45d1-8248-767b91a198fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:14.811998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecfd3a-f1a8-45d1-8248-767b91a198fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:14.812157 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecfd3a-f1a8-45d1-8248-767b91a198fa admin admin] PATH_INFO: `/v3/users/09ceeef2a0844fe2bef9bfb9b9c09deb` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:14.813700 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecfd3a-f1a8-45d1-8248-767b91a198fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=09ceeef2a0844fe2bef9bfb9b9c09deb)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:14.818936 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecfd3a-f1a8-45d1-8248-767b91a198fa admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:14.916641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f0ecfd3a-f1a8-45d1-8248-767b91a198fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09ceeef2a0844fe2bef9bfb9b9c09deb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:14.928563 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f0ecfd3a-f1a8-45d1-8248-767b91a198fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09ceeef2a0844fe2bef9bfb9b9c09deb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:14.929472 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f0ecfd3a-f1a8-45d1-8248-767b91a198fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09ceeef2a0844fe2bef9bfb9b9c09deb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:14.930121 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f0ecfd3a-f1a8-45d1-8248-767b91a198fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09ceeef2a0844fe2bef9bfb9b9c09deb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:14.937427 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:14.952199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f0ecfd3a-f1a8-45d1-8248-767b91a198fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09ceeef2a0844fe2bef9bfb9b9c09deb'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:14.971816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 983/1985] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:14 2022] DELETE /identity/v3/users/09ceeef2a0844fe2bef9bfb9b9c09deb => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:15.283911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7f4c6-699a-4a55-b8b0-9d824e18bcde admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:15.286883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-36f7f4c6-699a-4a55-b8b0-9d824e18bcde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:15.319288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7f4c6-699a-4a55-b8b0-9d824e18bcde admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:15.365680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:15.402675 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c317ab-a155-453b-a478-4efad9dfa503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:15.403611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c317ab-a155-453b-a478-4efad9dfa503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:15.403932 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c317ab-a155-453b-a478-4efad9dfa503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:15.404206 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c317ab-a155-453b-a478-4efad9dfa503 admin admin] PATH_INFO: `/v3/projects/99be19f7a21142108d05d355b21d6186` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:15.405328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c317ab-a155-453b-a478-4efad9dfa503 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99be19f7a21142108d05d355b21d6186)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:15.407215 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c317ab-a155-453b-a478-4efad9dfa503 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:15.458017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a6c317ab-a155-453b-a478-4efad9dfa503 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99be19f7a21142108d05d355b21d6186', 'request_id': 'req-a6c317ab-a155-453b-a478-4efad9dfa503'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:15.466440 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a6c317ab-a155-453b-a478-4efad9dfa503 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99be19f7a21142108d05d355b21d6186', 'request_id': 'req-a6c317ab-a155-453b-a478-4efad9dfa503'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:15.476569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a6c317ab-a155-453b-a478-4efad9dfa503 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99be19f7a21142108d05d355b21d6186', 'request_id': 'req-a6c317ab-a155-453b-a478-4efad9dfa503'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:15.490570 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a6c317ab-a155-453b-a478-4efad9dfa503 admin admin] The token cache is being invalidate because project 99be19f7a21142108d05d355b21d6186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:15.490861 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a6c317ab-a155-453b-a478-4efad9dfa503 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:15.491506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a6c317ab-a155-453b-a478-4efad9dfa503 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:15.493601 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1003/1986] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:15 2022] DELETE /identity/v3/projects/99be19f7a21142108d05d355b21d6186 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:15.498518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a172c2-cb5f-4013-9790-faf54485e4f8 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:15.498897 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a172c2-cb5f-4013-9790-faf54485e4f8 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:15.499244 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a172c2-cb5f-4013-9790-faf54485e4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:15.523583 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a7a172c2-cb5f-4013-9790-faf54485e4f8 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:15.524985 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a7a172c2-cb5f-4013-9790-faf54485e4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:15.592212 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 984/1987] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:15.601172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add42ff9-270b-41c1-9203-6eb49208cc9a None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:15.601489 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add42ff9-270b-41c1-9203-6eb49208cc9a None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:15.601766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add42ff9-270b-41c1-9203-6eb49208cc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:15.622786 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-add42ff9-270b-41c1-9203-6eb49208cc9a None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:15.624020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-add42ff9-270b-41c1-9203-6eb49208cc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:15.632563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1004/1988] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:15.637858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a172c2-cb5f-4013-9790-faf54485e4f8 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:15.649577 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a172c2-cb5f-4013-9790-faf54485e4f8 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:15.660139 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cb355-2f2d-458d-a403-d0eb98509191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:15.660916 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cb355-2f2d-458d-a403-d0eb98509191 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:15.661260 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cb355-2f2d-458d-a403-d0eb98509191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:15.661596 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cb355-2f2d-458d-a403-d0eb98509191 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:15.662319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1cb355-2f2d-458d-a403-d0eb98509191 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:15.662684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1cb355-2f2d-458d-a403-d0eb98509191 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:15.664819 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1cb355-2f2d-458d-a403-d0eb98509191 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:15.680895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 985/1989] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:15.685139 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add42ff9-270b-41c1-9203-6eb49208cc9a None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:15.695687 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add42ff9-270b-41c1-9203-6eb49208cc9a None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:15.702131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c72483-e67e-4523-abb3-42cfc5daaec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:15.702910 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c72483-e67e-4523-abb3-42cfc5daaec4 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:15.703156 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c72483-e67e-4523-abb3-42cfc5daaec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:15.703497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c72483-e67e-4523-abb3-42cfc5daaec4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:15.705326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c72483-e67e-4523-abb3-42cfc5daaec4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:15.710118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c72483-e67e-4523-abb3-42cfc5daaec4 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:15.752112 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1005/1990] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:15 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:15.755408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cb355-2f2d-458d-a403-d0eb98509191 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:15.776574 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cb355-2f2d-458d-a403-d0eb98509191 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:15.911797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c0a07-e528-4a35-aa90-20fc3435cd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:15.912681 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c0a07-e528-4a35-aa90-20fc3435cd96 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:15.913165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c0a07-e528-4a35-aa90-20fc3435cd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:15.913447 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c0a07-e528-4a35-aa90-20fc3435cd96 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:15.914090 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c0a07-e528-4a35-aa90-20fc3435cd96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:15.917540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c0a07-e528-4a35-aa90-20fc3435cd96 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:15.943860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 986/1991] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:15 2022] POST /identity/v3/users => generated 291 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:15.948901 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c72483-e67e-4523-abb3-42cfc5daaec4 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:15.976919 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c72483-e67e-4523-abb3-42cfc5daaec4 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:15.983542 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28535019-b2a2-4c3c-9699-59870fa2673c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:15.984296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28535019-b2a2-4c3c-9699-59870fa2673c admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:15.984880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28535019-b2a2-4c3c-9699-59870fa2673c admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:15.985346 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28535019-b2a2-4c3c-9699-59870fa2673c admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:15.986231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28535019-b2a2-4c3c-9699-59870fa2673c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:15.988237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28535019-b2a2-4c3c-9699-59870fa2673c admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:16.007591 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1006/1992] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:16.012018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c0a07-e528-4a35-aa90-20fc3435cd96 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:16.025912 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c0a07-e528-4a35-aa90-20fc3435cd96 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:16.034606 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133dfcf2-e46b-4d4e-b0b2-79cbc088a8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:16.035582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dfcf2-e46b-4d4e-b0b2-79cbc088a8aa admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:16.036097 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dfcf2-e46b-4d4e-b0b2-79cbc088a8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:16.036619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dfcf2-e46b-4d4e-b0b2-79cbc088a8aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:16.037606 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133dfcf2-e46b-4d4e-b0b2-79cbc088a8aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:16.039328 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133dfcf2-e46b-4d4e-b0b2-79cbc088a8aa admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:16.060777 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 987/1993] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:16.065113 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28535019-b2a2-4c3c-9699-59870fa2673c admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:16.077470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28535019-b2a2-4c3c-9699-59870fa2673c admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:16.083025 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c603c7e-8ce3-4d2d-8565-63d0cbac27ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:16.083673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c603c7e-8ce3-4d2d-8565-63d0cbac27ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:16.084248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c603c7e-8ce3-4d2d-8565-63d0cbac27ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:16.084606 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c603c7e-8ce3-4d2d-8565-63d0cbac27ad admin admin] PATH_INFO: `/v3/projects/da1983ab52ed49929361770ed46e8bfa/users/7ec4d7cc48164735ab80b302308e4038/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:16.104599 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c603c7e-8ce3-4d2d-8565-63d0cbac27ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=da1983ab52ed49929361770ed46e8bfa, user_id=7ec4d7cc48164735ab80b302308e4038, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:16.107247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c603c7e-8ce3-4d2d-8565-63d0cbac27ad admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:16.123007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1007/1994] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:16 2022] PUT /identity/v3/projects/da1983ab52ed49929361770ed46e8bfa/users/7ec4d7cc48164735ab80b302308e4038/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:16.134769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e5f3-321f-442e-9c02-1170ded571f6 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:16.135330 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e5f3-321f-442e-9c02-1170ded571f6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:16.137489 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e5f3-321f-442e-9c02-1170ded571f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:16.169758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9104e5f3-321f-442e-9c02-1170ded571f6 None None] MFA Rules not processed for user `7ec4d7cc48164735ab80b302308e4038`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:16.171053 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9104e5f3-321f-442e-9c02-1170ded571f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:16.202776 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:16.253101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 988/1995] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:16.257464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c603c7e-8ce3-4d2d-8565-63d0cbac27ad admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:16.398383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:16.414291 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c603c7e-8ce3-4d2d-8565-63d0cbac27ad admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:16.425920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd731f-cc88-4338-8c7b-5455166c8930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:16.426554 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd731f-cc88-4338-8c7b-5455166c8930 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:16.428807 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd731f-cc88-4338-8c7b-5455166c8930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:16.429214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd731f-cc88-4338-8c7b-5455166c8930 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:16.429862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd731f-cc88-4338-8c7b-5455166c8930 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:16.437980 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd731f-cc88-4338-8c7b-5455166c8930 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:16.500372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104e5f3-321f-442e-9c02-1170ded571f6 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:16.508636 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1008/1996] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:16 2022] POST /identity/v3/projects => generated 383 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:24:16.517964 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104e5f3-321f-442e-9c02-1170ded571f6 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:16.583359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549c5c3-4edd-40cb-890b-eaa4d57484e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:16.584254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549c5c3-4edd-40cb-890b-eaa4d57484e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:16.584555 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549c5c3-4edd-40cb-890b-eaa4d57484e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:16.584819 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549c5c3-4edd-40cb-890b-eaa4d57484e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:16.585441 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549c5c3-4edd-40cb-890b-eaa4d57484e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:16.587386 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549c5c3-4edd-40cb-890b-eaa4d57484e2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:16.612788 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 989/1997] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:16 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:16.614420 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd731f-cc88-4338-8c7b-5455166c8930 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:16.625619 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd731f-cc88-4338-8c7b-5455166c8930 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:16.631811 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61312b10-744c-4754-9ed1-a28f9fd65763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:16.632665 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61312b10-744c-4754-9ed1-a28f9fd65763 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:16.632993 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61312b10-744c-4754-9ed1-a28f9fd65763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:16.633263 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61312b10-744c-4754-9ed1-a28f9fd65763 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:16.634112 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61312b10-744c-4754-9ed1-a28f9fd65763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:16.636402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61312b10-744c-4754-9ed1-a28f9fd65763 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:16.650565 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1009/1998] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:16.656820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549c5c3-4edd-40cb-890b-eaa4d57484e2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:16.671786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549c5c3-4edd-40cb-890b-eaa4d57484e2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:16.677927 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd383c20-e78d-4514-b815-5c87dd94cc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:16.678687 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd383c20-e78d-4514-b815-5c87dd94cc13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:16.678963 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd383c20-e78d-4514-b815-5c87dd94cc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:16.679409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd383c20-e78d-4514-b815-5c87dd94cc13 admin admin] PATH_INFO: `/v3/projects/558c4ed182d64567a777b79ac1c80228/users/17f2cff6cf1a47f19f3f0b5208698884/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:16.703807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd383c20-e78d-4514-b815-5c87dd94cc13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=558c4ed182d64567a777b79ac1c80228, user_id=17f2cff6cf1a47f19f3f0b5208698884, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:16.706560 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd383c20-e78d-4514-b815-5c87dd94cc13 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:16.723564 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 990/1999] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:16 2022] PUT /identity/v3/projects/558c4ed182d64567a777b79ac1c80228/users/17f2cff6cf1a47f19f3f0b5208698884/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:16.724720 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61312b10-744c-4754-9ed1-a28f9fd65763 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:16.735286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61312b10-744c-4754-9ed1-a28f9fd65763 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:16.833613 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97365155-c6f4-4b11-a470-2c5467c7299c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:16.835365 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97365155-c6f4-4b11-a470-2c5467c7299c admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:16.835826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97365155-c6f4-4b11-a470-2c5467c7299c admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:16.836223 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97365155-c6f4-4b11-a470-2c5467c7299c admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:16.837842 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97365155-c6f4-4b11-a470-2c5467c7299c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:16.840643 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97365155-c6f4-4b11-a470-2c5467c7299c admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:16.854836 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1010/2000] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:24:16.859133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd383c20-e78d-4514-b815-5c87dd94cc13 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:16.870468 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd383c20-e78d-4514-b815-5c87dd94cc13 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:16.875341 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee0a7f-2f6f-46bc-b93f-074762f87cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:16.876026 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee0a7f-2f6f-46bc-b93f-074762f87cc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:16.876306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee0a7f-2f6f-46bc-b93f-074762f87cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:16.876574 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee0a7f-2f6f-46bc-b93f-074762f87cc8 admin admin] PATH_INFO: `/v3/projects/558c4ed182d64567a777b79ac1c80228/users/17f2cff6cf1a47f19f3f0b5208698884/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:16.878633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee0a7f-2f6f-46bc-b93f-074762f87cc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=558c4ed182d64567a777b79ac1c80228, user_id=17f2cff6cf1a47f19f3f0b5208698884, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:16.881275 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee0a7f-2f6f-46bc-b93f-074762f87cc8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:16.892770 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 991/2001] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:16 2022] PUT /identity/v3/projects/558c4ed182d64567a777b79ac1c80228/users/17f2cff6cf1a47f19f3f0b5208698884/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:16.904226 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd1583-0f03-460d-ad00-75e7294a8af0 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:16.904514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd1583-0f03-460d-ad00-75e7294a8af0 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:16.904788 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd1583-0f03-460d-ad00-75e7294a8af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:16.933417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-44bd1583-0f03-460d-ad00-75e7294a8af0 None None] MFA Rules not processed for user `17f2cff6cf1a47f19f3f0b5208698884`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:16.934665 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-44bd1583-0f03-460d-ad00-75e7294a8af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:17.006219 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1011/2002] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:17.018230 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee0a7f-2f6f-46bc-b93f-074762f87cc8 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:17.019967 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-06ee0a7f-2f6f-46bc-b93f-074762f87cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:17.079108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee0a7f-2f6f-46bc-b93f-074762f87cc8 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:17.082754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d7d41-dbf9-4fe6-9d3c-ae6ef41ad52a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:17.083351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d7d41-dbf9-4fe6-9d3c-ae6ef41ad52a service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:17.083626 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d7d41-dbf9-4fe6-9d3c-ae6ef41ad52a service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:17.084271 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d7d41-dbf9-4fe6-9d3c-ae6ef41ad52a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:17.093126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d7d41-dbf9-4fe6-9d3c-ae6ef41ad52a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:17.094566 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d7d41-dbf9-4fe6-9d3c-ae6ef41ad52a service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:17.104175 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 992/2003] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:17 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:17.204306 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd1583-0f03-460d-ad00-75e7294a8af0 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:17.216897 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd1583-0f03-460d-ad00-75e7294a8af0 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:17.220693 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9d794-cb51-4e15-831e-8e93eda2aafb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:17.221742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9d794-cb51-4e15-831e-8e93eda2aafb service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:17.222030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9d794-cb51-4e15-831e-8e93eda2aafb service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:17.222302 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9d794-cb51-4e15-831e-8e93eda2aafb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:17.277959 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:17.289555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd9d794-cb51-4e15-831e-8e93eda2aafb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:17.291044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd9d794-cb51-4e15-831e-8e93eda2aafb service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:17.300195 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1012/2004] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:17.550328 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d7d41-dbf9-4fe6-9d3c-ae6ef41ad52a service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:17.553717 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ee0d7d41-dbf9-4fe6-9d3c-ae6ef41ad52a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:17.613977 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:17.627205 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9d794-cb51-4e15-831e-8e93eda2aafb service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:17.657550 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d7d41-dbf9-4fe6-9d3c-ae6ef41ad52a service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:17.664972 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60486d3-a74c-418c-8437-7a5ea16e11b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:17.665891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60486d3-a74c-418c-8437-7a5ea16e11b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:17.666172 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60486d3-a74c-418c-8437-7a5ea16e11b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:17.666450 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60486d3-a74c-418c-8437-7a5ea16e11b4 admin admin] PATH_INFO: `/v3/users/e4b371a1fc944ad2a24b960e81d1cc30` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:17.667842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60486d3-a74c-418c-8437-7a5ea16e11b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4b371a1fc944ad2a24b960e81d1cc30)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:17.670148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60486d3-a74c-418c-8437-7a5ea16e11b4 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:17.753667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9d794-cb51-4e15-831e-8e93eda2aafb service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:17.767990 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af546d-cd56-4fdf-b151-29cba4c1085d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:17.773351 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af546d-cd56-4fdf-b151-29cba4c1085d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:17.773700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af546d-cd56-4fdf-b151-29cba4c1085d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:17.774987 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af546d-cd56-4fdf-b151-29cba4c1085d admin admin] PATH_INFO: `/v3/users/7ec4d7cc48164735ab80b302308e4038` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:17.777323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af546d-cd56-4fdf-b151-29cba4c1085d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ec4d7cc48164735ab80b302308e4038)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:17.780111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af546d-cd56-4fdf-b151-29cba4c1085d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:17.784484 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e60486d3-a74c-418c-8437-7a5ea16e11b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4b371a1fc944ad2a24b960e81d1cc30'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:17.810555 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e60486d3-a74c-418c-8437-7a5ea16e11b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4b371a1fc944ad2a24b960e81d1cc30'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:17.812366 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e60486d3-a74c-418c-8437-7a5ea16e11b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4b371a1fc944ad2a24b960e81d1cc30'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:17.813415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e60486d3-a74c-418c-8437-7a5ea16e11b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4b371a1fc944ad2a24b960e81d1cc30'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:17.829302 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e60486d3-a74c-418c-8437-7a5ea16e11b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4b371a1fc944ad2a24b960e81d1cc30'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:17.846453 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 993/2005] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:17 2022] DELETE /identity/v3/users/e4b371a1fc944ad2a24b960e81d1cc30 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:17.847539 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60486d3-a74c-418c-8437-7a5ea16e11b4 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:17.850201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e60486d3-a74c-418c-8437-7a5ea16e11b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:17.872183 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-03af546d-cd56-4fdf-b151-29cba4c1085d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ec4d7cc48164735ab80b302308e4038'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:17.873078 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-03af546d-cd56-4fdf-b151-29cba4c1085d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ec4d7cc48164735ab80b302308e4038'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:17.873911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-03af546d-cd56-4fdf-b151-29cba4c1085d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ec4d7cc48164735ab80b302308e4038'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:17.890173 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-03af546d-cd56-4fdf-b151-29cba4c1085d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ec4d7cc48164735ab80b302308e4038'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:17.911472 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-03af546d-cd56-4fdf-b151-29cba4c1085d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ec4d7cc48164735ab80b302308e4038'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:17.937058 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1013/2006] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:17 2022] DELETE /identity/v3/users/7ec4d7cc48164735ab80b302308e4038 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:17.937937 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af546d-cd56-4fdf-b151-29cba4c1085d admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:17.938075 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60486d3-a74c-418c-8437-7a5ea16e11b4 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:17.940221 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-03af546d-cd56-4fdf-b151-29cba4c1085d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:17.996257 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:18.004379 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7476637-ce88-436a-ae4d-3cead0a8d43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:18.005283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7476637-ce88-436a-ae4d-3cead0a8d43b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:18.005734 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7476637-ce88-436a-ae4d-3cead0a8d43b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:18.006169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7476637-ce88-436a-ae4d-3cead0a8d43b admin admin] PATH_INFO: `/v3/users/9d2a9b0e225147d9b9558d136763fe00` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:18.020639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7476637-ce88-436a-ae4d-3cead0a8d43b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d2a9b0e225147d9b9558d136763fe00)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:18.023333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7476637-ce88-436a-ae4d-3cead0a8d43b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:18.050026 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af546d-cd56-4fdf-b151-29cba4c1085d admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:18.060022 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34645c14-3e43-4107-9e00-815db1ad702e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:18.060943 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34645c14-3e43-4107-9e00-815db1ad702e service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:18.061752 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34645c14-3e43-4107-9e00-815db1ad702e service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:18.063193 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34645c14-3e43-4107-9e00-815db1ad702e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:18.067001 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-34645c14-3e43-4107-9e00-815db1ad702e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:18.117225 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a7476637-ce88-436a-ae4d-3cead0a8d43b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d2a9b0e225147d9b9558d136763fe00'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:18.125766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:18.139339 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a7476637-ce88-436a-ae4d-3cead0a8d43b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d2a9b0e225147d9b9558d136763fe00'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:18.140159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a7476637-ce88-436a-ae4d-3cead0a8d43b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d2a9b0e225147d9b9558d136763fe00'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:18.140774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a7476637-ce88-436a-ae4d-3cead0a8d43b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d2a9b0e225147d9b9558d136763fe00'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:18.171608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a7476637-ce88-436a-ae4d-3cead0a8d43b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d2a9b0e225147d9b9558d136763fe00'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:18.200808 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 994/2007] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:17 2022] DELETE /identity/v3/users/9d2a9b0e225147d9b9558d136763fe00 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:18.202601 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf52b4-bd7b-46f4-b15c-2053082660c8 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:18.205098 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf52b4-bd7b-46f4-b15c-2053082660c8 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:18.205909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf52b4-bd7b-46f4-b15c-2053082660c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:18.206815 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34645c14-3e43-4107-9e00-815db1ad702e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:18.208960 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34645c14-3e43-4107-9e00-815db1ad702e service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:18.233126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-4cbf52b4-bd7b-46f4-b15c-2053082660c8 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:18.235112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4cbf52b4-bd7b-46f4-b15c-2053082660c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:18.253006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1014/2008] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:18.254662 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9661ba-9172-4af1-80e5-99f9f6891741 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:18.256870 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9661ba-9172-4af1-80e5-99f9f6891741 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:18.259909 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9661ba-9172-4af1-80e5-99f9f6891741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:18.307105 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-5c9661ba-9172-4af1-80e5-99f9f6891741 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:18.308484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5c9661ba-9172-4af1-80e5-99f9f6891741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:18.314282 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 995/2009] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:18.315953 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a19671-551c-43dc-9edf-339094e626bf None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:18.316453 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a19671-551c-43dc-9edf-339094e626bf None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:18.316938 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a19671-551c-43dc-9edf-339094e626bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:18.332135 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1015/2010] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:18.333555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9661ba-9172-4af1-80e5-99f9f6891741 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:18.335435 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5c9661ba-9172-4af1-80e5-99f9f6891741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:18.358212 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-23a19671-551c-43dc-9edf-339094e626bf None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:18.360843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-23a19671-551c-43dc-9edf-339094e626bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:18.384122 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 996/2011] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:18.386083 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a19671-551c-43dc-9edf-339094e626bf None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:18.389826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-23a19671-551c-43dc-9edf-339094e626bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:18.429734 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9661ba-9172-4af1-80e5-99f9f6891741 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:18.440461 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a0e03-3322-4fde-8a70-f027097b0f56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:18.441627 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a0e03-3322-4fde-8a70-f027097b0f56 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:18.452993 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a0e03-3322-4fde-8a70-f027097b0f56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:18.453427 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a0e03-3322-4fde-8a70-f027097b0f56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:18.476952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a0e03-3322-4fde-8a70-f027097b0f56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:18.479101 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a0e03-3322-4fde-8a70-f027097b0f56 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:18.492457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a19671-551c-43dc-9edf-339094e626bf None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:18.494635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1016/2012] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:18.496312 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a0e03-3322-4fde-8a70-f027097b0f56 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:18.497112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e2be2-bb01-47f0-8de7-92d737e17c85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:18.497868 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e2be2-bb01-47f0-8de7-92d737e17c85 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:18.498150 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e2be2-bb01-47f0-8de7-92d737e17c85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:18.498451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e2be2-bb01-47f0-8de7-92d737e17c85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:18.507737 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a0e03-3322-4fde-8a70-f027097b0f56 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:18.512613 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a337ac-3145-49a3-b4c9-0f1810796780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:18.513303 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a337ac-3145-49a3-b4c9-0f1810796780 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:18.513503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e2be2-bb01-47f0-8de7-92d737e17c85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:18.513797 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a337ac-3145-49a3-b4c9-0f1810796780 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:18.514137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a337ac-3145-49a3-b4c9-0f1810796780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:18.515959 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e2be2-bb01-47f0-8de7-92d737e17c85 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:18.528260 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a337ac-3145-49a3-b4c9-0f1810796780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:18.530200 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a337ac-3145-49a3-b4c9-0f1810796780 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:18.532572 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 997/2013] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:18.551304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1017/2014] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:19.148062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e2be2-bb01-47f0-8de7-92d737e17c85 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:19.149557 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b78e2be2-bb01-47f0-8de7-92d737e17c85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:19.164226 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e2be2-bb01-47f0-8de7-92d737e17c85 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:19.181560 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99389891-bc4c-4a4e-bfc2-89e71166101a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:19.182241 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99389891-bc4c-4a4e-bfc2-89e71166101a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:19.182551 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99389891-bc4c-4a4e-bfc2-89e71166101a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:19.182791 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99389891-bc4c-4a4e-bfc2-89e71166101a admin admin] PATH_INFO: `/v3/projects/a72e725b681e4561aa7764b426c0910d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:19.184617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99389891-bc4c-4a4e-bfc2-89e71166101a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a72e725b681e4561aa7764b426c0910d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:19.186584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99389891-bc4c-4a4e-bfc2-89e71166101a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:19.189687 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:19.252136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-99389891-bc4c-4a4e-bfc2-89e71166101a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a72e725b681e4561aa7764b426c0910d', 'request_id': 'req-99389891-bc4c-4a4e-bfc2-89e71166101a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.261664 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-99389891-bc4c-4a4e-bfc2-89e71166101a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a72e725b681e4561aa7764b426c0910d', 'request_id': 'req-99389891-bc4c-4a4e-bfc2-89e71166101a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.267910 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-99389891-bc4c-4a4e-bfc2-89e71166101a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a72e725b681e4561aa7764b426c0910d', 'request_id': 'req-99389891-bc4c-4a4e-bfc2-89e71166101a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.287910 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-99389891-bc4c-4a4e-bfc2-89e71166101a admin admin] The token cache is being invalidate because project a72e725b681e4561aa7764b426c0910d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:19.288232 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-99389891-bc4c-4a4e-bfc2-89e71166101a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.288844 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-99389891-bc4c-4a4e-bfc2-89e71166101a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.290610 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 998/2015] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:19 2022] DELETE /identity/v3/projects/a72e725b681e4561aa7764b426c0910d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:19.294984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a337ac-3145-49a3-b4c9-0f1810796780 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:19.315569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a337ac-3145-49a3-b4c9-0f1810796780 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:19.392933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899f3c0-3ea6-45b3-8d27-876dc3f4b103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:19.393920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899f3c0-3ea6-45b3-8d27-876dc3f4b103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:19.394355 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899f3c0-3ea6-45b3-8d27-876dc3f4b103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:19.394920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899f3c0-3ea6-45b3-8d27-876dc3f4b103 admin admin] PATH_INFO: `/v3/users/09f5fdd3849449c192e16d8091348f16` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:19.396321 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899f3c0-3ea6-45b3-8d27-876dc3f4b103 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09f5fdd3849449c192e16d8091348f16)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:19.398422 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899f3c0-3ea6-45b3-8d27-876dc3f4b103 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:19.418049 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99389891-bc4c-4a4e-bfc2-89e71166101a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:19.419609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-99389891-bc4c-4a4e-bfc2-89e71166101a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:19.443799 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99389891-bc4c-4a4e-bfc2-89e71166101a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:19.477443 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663ca10-a922-4189-bb8c-b9cda912102b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:19.478140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663ca10-a922-4189-bb8c-b9cda912102b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:19.478430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663ca10-a922-4189-bb8c-b9cda912102b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:19.478724 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663ca10-a922-4189-bb8c-b9cda912102b admin admin] PATH_INFO: `/v3/projects/330f59a1416f45508637bd01631cb14c` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:19.479979 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663ca10-a922-4189-bb8c-b9cda912102b admin admin] RBAC: Authorizing `identity:delete_project(project_id=330f59a1416f45508637bd01631cb14c)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:19.481914 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663ca10-a922-4189-bb8c-b9cda912102b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:19.486492 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c899f3c0-3ea6-45b3-8d27-876dc3f4b103 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09f5fdd3849449c192e16d8091348f16'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.487431 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c899f3c0-3ea6-45b3-8d27-876dc3f4b103 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09f5fdd3849449c192e16d8091348f16'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.488261 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c899f3c0-3ea6-45b3-8d27-876dc3f4b103 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09f5fdd3849449c192e16d8091348f16'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.489945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:19.509841 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c899f3c0-3ea6-45b3-8d27-876dc3f4b103 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09f5fdd3849449c192e16d8091348f16'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.536367 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c899f3c0-3ea6-45b3-8d27-876dc3f4b103 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09f5fdd3849449c192e16d8091348f16'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.543483 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b663ca10-a922-4189-bb8c-b9cda912102b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '330f59a1416f45508637bd01631cb14c', 'request_id': 'req-b663ca10-a922-4189-bb8c-b9cda912102b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.552856 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b663ca10-a922-4189-bb8c-b9cda912102b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '330f59a1416f45508637bd01631cb14c', 'request_id': 'req-b663ca10-a922-4189-bb8c-b9cda912102b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.561183 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b663ca10-a922-4189-bb8c-b9cda912102b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '330f59a1416f45508637bd01631cb14c', 'request_id': 'req-b663ca10-a922-4189-bb8c-b9cda912102b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.563946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1018/2016] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:19 2022] DELETE /identity/v3/users/09f5fdd3849449c192e16d8091348f16 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:19.565036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899f3c0-3ea6-45b3-8d27-876dc3f4b103 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:19.567597 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c899f3c0-3ea6-45b3-8d27-876dc3f4b103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:19.588662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b663ca10-a922-4189-bb8c-b9cda912102b admin admin] The token cache is being invalidate because project 330f59a1416f45508637bd01631cb14c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:19.589385 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b663ca10-a922-4189-bb8c-b9cda912102b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.590745 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b663ca10-a922-4189-bb8c-b9cda912102b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.592745 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 999/2017] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:19 2022] DELETE /identity/v3/projects/330f59a1416f45508637bd01631cb14c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:19.597491 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663ca10-a922-4189-bb8c-b9cda912102b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:19.599685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b663ca10-a922-4189-bb8c-b9cda912102b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:19.671225 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899f3c0-3ea6-45b3-8d27-876dc3f4b103 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:19.677779 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0056ca9-801b-48ae-a9d8-04d65c06c946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:19.678431 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0056ca9-801b-48ae-a9d8-04d65c06c946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:19.678722 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0056ca9-801b-48ae-a9d8-04d65c06c946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:19.678993 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0056ca9-801b-48ae-a9d8-04d65c06c946 admin admin] PATH_INFO: `/v3/projects/da1983ab52ed49929361770ed46e8bfa` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:19.681040 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0056ca9-801b-48ae-a9d8-04d65c06c946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da1983ab52ed49929361770ed46e8bfa)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:19.683667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0056ca9-801b-48ae-a9d8-04d65c06c946 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:19.732310 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663ca10-a922-4189-bb8c-b9cda912102b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:19.741385 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b40868-5ee8-4028-bc71-c30724a1adb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:19.742402 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b40868-5ee8-4028-bc71-c30724a1adb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:19.743576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b40868-5ee8-4028-bc71-c30724a1adb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:19.744213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b40868-5ee8-4028-bc71-c30724a1adb3 admin admin] PATH_INFO: `/v3/users/58142d0708e349298ca9cd5f069be3bd` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:19.745913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b40868-5ee8-4028-bc71-c30724a1adb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58142d0708e349298ca9cd5f069be3bd)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:19.748599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b40868-5ee8-4028-bc71-c30724a1adb3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:19.776674 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:19.782207 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e0056ca9-801b-48ae-a9d8-04d65c06c946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da1983ab52ed49929361770ed46e8bfa', 'request_id': 'req-e0056ca9-801b-48ae-a9d8-04d65c06c946'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.800289 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e0056ca9-801b-48ae-a9d8-04d65c06c946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da1983ab52ed49929361770ed46e8bfa', 'request_id': 'req-e0056ca9-801b-48ae-a9d8-04d65c06c946'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.811823 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e0056ca9-801b-48ae-a9d8-04d65c06c946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da1983ab52ed49929361770ed46e8bfa', 'request_id': 'req-e0056ca9-801b-48ae-a9d8-04d65c06c946'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.863013 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e0056ca9-801b-48ae-a9d8-04d65c06c946 admin admin] The token cache is being invalidate because project da1983ab52ed49929361770ed46e8bfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:19.863338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e0056ca9-801b-48ae-a9d8-04d65c06c946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.865860 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e0056ca9-801b-48ae-a9d8-04d65c06c946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.868516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1019/2018] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:19 2022] DELETE /identity/v3/projects/da1983ab52ed49929361770ed46e8bfa => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:19.872351 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0056ca9-801b-48ae-a9d8-04d65c06c946 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:19.873246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a2b40868-5ee8-4028-bc71-c30724a1adb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58142d0708e349298ca9cd5f069be3bd'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.884089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a2b40868-5ee8-4028-bc71-c30724a1adb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58142d0708e349298ca9cd5f069be3bd'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.884854 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a2b40868-5ee8-4028-bc71-c30724a1adb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58142d0708e349298ca9cd5f069be3bd'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.885602 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a2b40868-5ee8-4028-bc71-c30724a1adb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58142d0708e349298ca9cd5f069be3bd'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.886645 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e0056ca9-801b-48ae-a9d8-04d65c06c946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:19.903214 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a2b40868-5ee8-4028-bc71-c30724a1adb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58142d0708e349298ca9cd5f069be3bd'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:19.920410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1000/2019] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:19 2022] DELETE /identity/v3/users/58142d0708e349298ca9cd5f069be3bd => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:20.016796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:20.024509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0056ca9-801b-48ae-a9d8-04d65c06c946 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:20.158003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a56a21-6f51-4520-a345-0d06dcea20f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:20.158752 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a56a21-6f51-4520-a345-0d06dcea20f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:20.159135 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a56a21-6f51-4520-a345-0d06dcea20f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:20.159850 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a56a21-6f51-4520-a345-0d06dcea20f6 admin admin] PATH_INFO: `/v3/users/17f2cff6cf1a47f19f3f0b5208698884` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:20.161049 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a56a21-6f51-4520-a345-0d06dcea20f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17f2cff6cf1a47f19f3f0b5208698884)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:20.163272 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a56a21-6f51-4520-a345-0d06dcea20f6 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:20.169608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b40868-5ee8-4028-bc71-c30724a1adb3 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:20.171631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a2b40868-5ee8-4028-bc71-c30724a1adb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:20.198429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b40868-5ee8-4028-bc71-c30724a1adb3 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:20.207201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d549e-0eea-4ed9-b8f4-41f2f0ada2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:20.208480 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d549e-0eea-4ed9-b8f4-41f2f0ada2f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:20.209197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d549e-0eea-4ed9-b8f4-41f2f0ada2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:20.209487 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d549e-0eea-4ed9-b8f4-41f2f0ada2f8 admin admin] PATH_INFO: `/v3/projects/97e8a82e81ec4fc5bd40354c4bf02b39` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:20.210729 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d549e-0eea-4ed9-b8f4-41f2f0ada2f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97e8a82e81ec4fc5bd40354c4bf02b39)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:20.213251 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d549e-0eea-4ed9-b8f4-41f2f0ada2f8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:20.239826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-69a56a21-6f51-4520-a345-0d06dcea20f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17f2cff6cf1a47f19f3f0b5208698884'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:20.240928 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-69a56a21-6f51-4520-a345-0d06dcea20f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17f2cff6cf1a47f19f3f0b5208698884'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:20.241788 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-69a56a21-6f51-4520-a345-0d06dcea20f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17f2cff6cf1a47f19f3f0b5208698884'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:20.252575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-69a56a21-6f51-4520-a345-0d06dcea20f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17f2cff6cf1a47f19f3f0b5208698884'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:20.267228 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-69a56a21-6f51-4520-a345-0d06dcea20f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17f2cff6cf1a47f19f3f0b5208698884'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:20.282336 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fc3d549e-0eea-4ed9-b8f4-41f2f0ada2f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97e8a82e81ec4fc5bd40354c4bf02b39', 'request_id': 'req-fc3d549e-0eea-4ed9-b8f4-41f2f0ada2f8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:20.284766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1020/2020] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:19 2022] DELETE /identity/v3/users/17f2cff6cf1a47f19f3f0b5208698884 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:20.291555 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fc3d549e-0eea-4ed9-b8f4-41f2f0ada2f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97e8a82e81ec4fc5bd40354c4bf02b39', 'request_id': 'req-fc3d549e-0eea-4ed9-b8f4-41f2f0ada2f8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:20.300272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fc3d549e-0eea-4ed9-b8f4-41f2f0ada2f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97e8a82e81ec4fc5bd40354c4bf02b39', 'request_id': 'req-fc3d549e-0eea-4ed9-b8f4-41f2f0ada2f8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:20.330437 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a56a21-6f51-4520-a345-0d06dcea20f6 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:20.333943 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-69a56a21-6f51-4520-a345-0d06dcea20f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:20.380457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fc3d549e-0eea-4ed9-b8f4-41f2f0ada2f8 admin admin] The token cache is being invalidate because project 97e8a82e81ec4fc5bd40354c4bf02b39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:20.380809 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fc3d549e-0eea-4ed9-b8f4-41f2f0ada2f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:20.384395 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fc3d549e-0eea-4ed9-b8f4-41f2f0ada2f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:20.392350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1001/2021] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:20 2022] DELETE /identity/v3/projects/97e8a82e81ec4fc5bd40354c4bf02b39 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:20.399593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d549e-0eea-4ed9-b8f4-41f2f0ada2f8 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:20.413708 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-fc3d549e-0eea-4ed9-b8f4-41f2f0ada2f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:20.567156 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d549e-0eea-4ed9-b8f4-41f2f0ada2f8 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:20.578777 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f184f6-4aa7-484e-8246-e88793b998e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:20.593263 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a56a21-6f51-4520-a345-0d06dcea20f6 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:20.595953 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f184f6-4aa7-484e-8246-e88793b998e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:20.597781 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f184f6-4aa7-484e-8246-e88793b998e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:20.598360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f184f6-4aa7-484e-8246-e88793b998e5 admin admin] PATH_INFO: `/v3/users/c748bbbe47564bfa9d67d587dd83b704` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:20.601414 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f184f6-4aa7-484e-8246-e88793b998e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c748bbbe47564bfa9d67d587dd83b704)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:20.606369 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49566b06-9646-4410-9b62-e217c87fc7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:20.607691 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49566b06-9646-4410-9b62-e217c87fc7c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:20.608073 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49566b06-9646-4410-9b62-e217c87fc7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:20.609247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49566b06-9646-4410-9b62-e217c87fc7c4 admin admin] PATH_INFO: `/v3/projects/90893c26b4b04527a58a61f894d91396` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:20.610639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f184f6-4aa7-484e-8246-e88793b998e5 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:20.612223 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49566b06-9646-4410-9b62-e217c87fc7c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90893c26b4b04527a58a61f894d91396)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:20.617412 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49566b06-9646-4410-9b62-e217c87fc7c4 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:20.681858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:20.691317 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:20.747862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-43f184f6-4aa7-484e-8246-e88793b998e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c748bbbe47564bfa9d67d587dd83b704'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:20.752319 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-49566b06-9646-4410-9b62-e217c87fc7c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90893c26b4b04527a58a61f894d91396', 'request_id': 'req-49566b06-9646-4410-9b62-e217c87fc7c4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:20.762293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-49566b06-9646-4410-9b62-e217c87fc7c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90893c26b4b04527a58a61f894d91396', 'request_id': 'req-49566b06-9646-4410-9b62-e217c87fc7c4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:20.769844 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-43f184f6-4aa7-484e-8246-e88793b998e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c748bbbe47564bfa9d67d587dd83b704'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:20.772403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-43f184f6-4aa7-484e-8246-e88793b998e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c748bbbe47564bfa9d67d587dd83b704'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:20.774581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-43f184f6-4aa7-484e-8246-e88793b998e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c748bbbe47564bfa9d67d587dd83b704'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:20.781474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-49566b06-9646-4410-9b62-e217c87fc7c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90893c26b4b04527a58a61f894d91396', 'request_id': 'req-49566b06-9646-4410-9b62-e217c87fc7c4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:20.799052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-43f184f6-4aa7-484e-8246-e88793b998e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c748bbbe47564bfa9d67d587dd83b704'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:20.806023 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-49566b06-9646-4410-9b62-e217c87fc7c4 admin admin] The token cache is being invalidate because project 90893c26b4b04527a58a61f894d91396 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:20.806538 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-49566b06-9646-4410-9b62-e217c87fc7c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:20.807703 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-49566b06-9646-4410-9b62-e217c87fc7c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:20.820091 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1021/2022] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:20 2022] DELETE /identity/v3/projects/90893c26b4b04527a58a61f894d91396 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:20.826192 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a48e4-d591-4d63-8ddb-10c60aed9bef None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:20.826709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a48e4-d591-4d63-8ddb-10c60aed9bef None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:20.827028 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a48e4-d591-4d63-8ddb-10c60aed9bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:20.836957 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1002/2023] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:20 2022] DELETE /identity/v3/users/c748bbbe47564bfa9d67d587dd83b704 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:20.862257 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-768a48e4-d591-4d63-8ddb-10c60aed9bef None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:20.863894 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-768a48e4-d591-4d63-8ddb-10c60aed9bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:21.009000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1022/2024] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:24:21.017429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd9b74-9bc3-439c-bdad-7e38aaadab70 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:21.017771 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd9b74-9bc3-439c-bdad-7e38aaadab70 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:21.018073 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd9b74-9bc3-439c-bdad-7e38aaadab70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:21.049953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a48e4-d591-4d63-8ddb-10c60aed9bef None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:21.051399 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-768a48e4-d591-4d63-8ddb-10c60aed9bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:21.054877 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-10fd9b74-9bc3-439c-bdad-7e38aaadab70 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:21.057485 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-10fd9b74-9bc3-439c-bdad-7e38aaadab70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:21.073805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a48e4-d591-4d63-8ddb-10c60aed9bef None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:21.073981 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1003/2025] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:21.074713 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd9b74-9bc3-439c-bdad-7e38aaadab70 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:21.087528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd9b74-9bc3-439c-bdad-7e38aaadab70 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:21.090386 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ebceb-9bd0-4643-a93b-b040098e7e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:21.091450 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ebceb-9bd0-4643-a93b-b040098e7e61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:21.091726 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ebceb-9bd0-4643-a93b-b040098e7e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:21.092619 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ebceb-9bd0-4643-a93b-b040098e7e61 admin admin] PATH_INFO: `/v3/users/1cd9db584be246bebcc6a3cb494aacb6` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:21.094764 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ebceb-9bd0-4643-a93b-b040098e7e61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cd9db584be246bebcc6a3cb494aacb6)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:21.096772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ebceb-9bd0-4643-a93b-b040098e7e61 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:21.110709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a8246-b98e-456c-8181-3bb222364800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:21.113679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a8246-b98e-456c-8181-3bb222364800 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:21.113679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a8246-b98e-456c-8181-3bb222364800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:21.113932 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a8246-b98e-456c-8181-3bb222364800 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:21.114600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a8246-b98e-456c-8181-3bb222364800 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:21.114908 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a8246-b98e-456c-8181-3bb222364800 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:21.120611 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a8246-b98e-456c-8181-3bb222364800 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:21.145989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1004/2026] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:24:21.145989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a8246-b98e-456c-8181-3bb222364800 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:21.147661 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3f5a8246-b98e-456c-8181-3bb222364800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:21.165613 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a8246-b98e-456c-8181-3bb222364800 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:21.184426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5aa86-f051-472d-9d2a-83152ee4102f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:21.185851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5aa86-f051-472d-9d2a-83152ee4102f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:21.186519 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5aa86-f051-472d-9d2a-83152ee4102f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:21.186519 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5aa86-f051-472d-9d2a-83152ee4102f admin admin] PATH_INFO: `/v3/projects/558c4ed182d64567a777b79ac1c80228` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:21.188400 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5aa86-f051-472d-9d2a-83152ee4102f admin admin] RBAC: Authorizing `identity:delete_project(project_id=558c4ed182d64567a777b79ac1c80228)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:21.190663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5aa86-f051-472d-9d2a-83152ee4102f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:21.199457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a49ebceb-9bd0-4643-a93b-b040098e7e61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cd9db584be246bebcc6a3cb494aacb6'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:21.202770 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a49ebceb-9bd0-4643-a93b-b040098e7e61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cd9db584be246bebcc6a3cb494aacb6'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:21.205002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a49ebceb-9bd0-4643-a93b-b040098e7e61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cd9db584be246bebcc6a3cb494aacb6'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:21.235592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a49ebceb-9bd0-4643-a93b-b040098e7e61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cd9db584be246bebcc6a3cb494aacb6'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:21.257026 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e5b5aa86-f051-472d-9d2a-83152ee4102f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '558c4ed182d64567a777b79ac1c80228', 'request_id': 'req-e5b5aa86-f051-472d-9d2a-83152ee4102f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:21.260498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a49ebceb-9bd0-4643-a93b-b040098e7e61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cd9db584be246bebcc6a3cb494aacb6'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:21.287923 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e5b5aa86-f051-472d-9d2a-83152ee4102f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '558c4ed182d64567a777b79ac1c80228', 'request_id': 'req-e5b5aa86-f051-472d-9d2a-83152ee4102f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:21.291836 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1023/2027] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:21 2022] DELETE /identity/v3/users/1cd9db584be246bebcc6a3cb494aacb6 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:21.293236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ebceb-9bd0-4643-a93b-b040098e7e61 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:21.296611 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e5b5aa86-f051-472d-9d2a-83152ee4102f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '558c4ed182d64567a777b79ac1c80228', 'request_id': 'req-e5b5aa86-f051-472d-9d2a-83152ee4102f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:21.298694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a49ebceb-9bd0-4643-a93b-b040098e7e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:21.312293 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e5b5aa86-f051-472d-9d2a-83152ee4102f admin admin] The token cache is being invalidate because project 558c4ed182d64567a777b79ac1c80228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:21.312732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e5b5aa86-f051-472d-9d2a-83152ee4102f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:21.314157 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e5b5aa86-f051-472d-9d2a-83152ee4102f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:21.316628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1005/2028] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:21 2022] DELETE /identity/v3/projects/558c4ed182d64567a777b79ac1c80228 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:21.317636 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5aa86-f051-472d-9d2a-83152ee4102f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:21.320059 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e5b5aa86-f051-472d-9d2a-83152ee4102f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:21.348311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:21.405696 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ebceb-9bd0-4643-a93b-b040098e7e61 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:21.408869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5aa86-f051-472d-9d2a-83152ee4102f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:21.412305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff50e34-9ef9-4097-a292-e65e6d438191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:21.413348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff50e34-9ef9-4097-a292-e65e6d438191 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:21.413780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff50e34-9ef9-4097-a292-e65e6d438191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:21.414215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c4774-f213-4c9e-9b6f-1278ec746589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:21.414346 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff50e34-9ef9-4097-a292-e65e6d438191 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:21.415418 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c4774-f213-4c9e-9b6f-1278ec746589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:21.415702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c4774-f213-4c9e-9b6f-1278ec746589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:21.415990 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff50e34-9ef9-4097-a292-e65e6d438191 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:21.416102 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c4774-f213-4c9e-9b6f-1278ec746589 admin admin] PATH_INFO: `/v3/projects/cea2ee65ace7429490a6dbb458953fc9` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:21.417735 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c4774-f213-4c9e-9b6f-1278ec746589 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cea2ee65ace7429490a6dbb458953fc9)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:21.418582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff50e34-9ef9-4097-a292-e65e6d438191 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:21.420034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c4774-f213-4c9e-9b6f-1278ec746589 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:21.421791 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:21.460014 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1024/2029] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:21 2022] POST /identity/v3/projects => generated 383 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:21.461146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dbfb0-cc46-4a93-870c-b5d24fffd2c7 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:21.462067 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dbfb0-cc46-4a93-870c-b5d24fffd2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:21.462622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dbfb0-cc46-4a93-870c-b5d24fffd2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:21.485226 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-2c6dbfb0-cc46-4a93-870c-b5d24fffd2c7 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:21.486569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2c6dbfb0-cc46-4a93-870c-b5d24fffd2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:21.506128 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a91c4774-f213-4c9e-9b6f-1278ec746589 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cea2ee65ace7429490a6dbb458953fc9', 'request_id': 'req-a91c4774-f213-4c9e-9b6f-1278ec746589'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:21.513516 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a91c4774-f213-4c9e-9b6f-1278ec746589 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cea2ee65ace7429490a6dbb458953fc9', 'request_id': 'req-a91c4774-f213-4c9e-9b6f-1278ec746589'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:21.520219 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a91c4774-f213-4c9e-9b6f-1278ec746589 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cea2ee65ace7429490a6dbb458953fc9', 'request_id': 'req-a91c4774-f213-4c9e-9b6f-1278ec746589'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:21.545760 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a91c4774-f213-4c9e-9b6f-1278ec746589 admin admin] The token cache is being invalidate because project cea2ee65ace7429490a6dbb458953fc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:21.546059 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a91c4774-f213-4c9e-9b6f-1278ec746589 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:21.547307 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a91c4774-f213-4c9e-9b6f-1278ec746589 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:21.553460 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1006/2030] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:21 2022] DELETE /identity/v3/projects/cea2ee65ace7429490a6dbb458953fc9 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:21.553460 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892953ab-8359-4417-baf0-3a61775f2ec0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:21.553460 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892953ab-8359-4417-baf0-3a61775f2ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:21.553460 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892953ab-8359-4417-baf0-3a61775f2ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:21.583579 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-892953ab-8359-4417-baf0-3a61775f2ec0 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:21.585588 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-892953ab-8359-4417-baf0-3a61775f2ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:21.589739 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1025/2031] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:21.589739 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6dbfb0-cc46-4a93-870c-b5d24fffd2c7 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:21.589739 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2c6dbfb0-cc46-4a93-870c-b5d24fffd2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:21.601319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1007/2032] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:21.601319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee08b25-2428-460e-a5c3-c2ce37c3bd6f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:21.601319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee08b25-2428-460e-a5c3-c2ce37c3bd6f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:21.601319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee08b25-2428-460e-a5c3-c2ce37c3bd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:21.603709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6dbfb0-cc46-4a93-870c-b5d24fffd2c7 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:21.612377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6a28f-31fb-418c-839b-789257b7abce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:21.612836 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6a28f-31fb-418c-839b-789257b7abce admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:21.613306 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6a28f-31fb-418c-839b-789257b7abce admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:21.613903 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6a28f-31fb-418c-839b-789257b7abce admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:21.614667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6a28f-31fb-418c-839b-789257b7abce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:21.617402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6a28f-31fb-418c-839b-789257b7abce admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:21.621123 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-4ee08b25-2428-460e-a5c3-c2ce37c3bd6f None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:21.622747 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4ee08b25-2428-460e-a5c3-c2ce37c3bd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:21.633492 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1008/2033] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:21.634706 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee08b25-2428-460e-a5c3-c2ce37c3bd6f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:21.636563 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4ee08b25-2428-460e-a5c3-c2ce37c3bd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:21.660923 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1026/2034] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:21 2022] POST /identity/v3/users => generated 292 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:21.662278 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1626107-713e-44ef-b612-af1b2e3fa012 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:21.662708 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1626107-713e-44ef-b612-af1b2e3fa012 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:21.662984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1626107-713e-44ef-b612-af1b2e3fa012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:21.700219 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-e1626107-713e-44ef-b612-af1b2e3fa012 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:21.702463 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e1626107-713e-44ef-b612-af1b2e3fa012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:21.714352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1027/2035] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:21.716305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6f080-5268-4bcf-a18b-cf2140f1b264 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:21.716596 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6f080-5268-4bcf-a18b-cf2140f1b264 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:21.717232 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6f080-5268-4bcf-a18b-cf2140f1b264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:21.736572 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee08b25-2428-460e-a5c3-c2ce37c3bd6f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:21.742222 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdd488-95e9-4195-8c4c-47ce15c9f04f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:21.743135 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdd488-95e9-4195-8c4c-47ce15c9f04f service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:21.743485 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdd488-95e9-4195-8c4c-47ce15c9f04f service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:21.743851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdd488-95e9-4195-8c4c-47ce15c9f04f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:21.745382 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-99a6f080-5268-4bcf-a18b-cf2140f1b264 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:21.748108 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-99a6f080-5268-4bcf-a18b-cf2140f1b264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:21.758486 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1028/2036] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:21.759706 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6f080-5268-4bcf-a18b-cf2140f1b264 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:21.769460 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efdd488-95e9-4195-8c4c-47ce15c9f04f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:21.770929 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efdd488-95e9-4195-8c4c-47ce15c9f04f service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:21.771355 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6f080-5268-4bcf-a18b-cf2140f1b264 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:21.779328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b2b02-2e5a-4b65-9d55-1a68d3092526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:21.780220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b2b02-2e5a-4b65-9d55-1a68d3092526 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:21.780675 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b2b02-2e5a-4b65-9d55-1a68d3092526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:21.781155 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b2b02-2e5a-4b65-9d55-1a68d3092526 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:21.781941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b2b02-2e5a-4b65-9d55-1a68d3092526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:21.782309 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1009/2037] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:21.782958 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdd488-95e9-4195-8c4c-47ce15c9f04f service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:21.784673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b2b02-2e5a-4b65-9d55-1a68d3092526 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:21.795425 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdd488-95e9-4195-8c4c-47ce15c9f04f service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:21.804166 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1029/2038] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:21.805662 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b2b02-2e5a-4b65-9d55-1a68d3092526 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:21.814109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50319b31-5487-4587-ac5b-f69c659c230f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:21.817179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50319b31-5487-4587-ac5b-f69c659c230f admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:21.817470 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50319b31-5487-4587-ac5b-f69c659c230f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:21.818129 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50319b31-5487-4587-ac5b-f69c659c230f admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:21.818810 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50319b31-5487-4587-ac5b-f69c659c230f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:21.819117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50319b31-5487-4587-ac5b-f69c659c230f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:21.820161 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b2b02-2e5a-4b65-9d55-1a68d3092526 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:21.821625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50319b31-5487-4587-ac5b-f69c659c230f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:21.828625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071fa36-8fc9-47f8-86cf-0d0fe89f8156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:21.829470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071fa36-8fc9-47f8-86cf-0d0fe89f8156 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:21.829758 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071fa36-8fc9-47f8-86cf-0d0fe89f8156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:21.830536 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071fa36-8fc9-47f8-86cf-0d0fe89f8156 admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:21.830797 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071fa36-8fc9-47f8-86cf-0d0fe89f8156 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:21.831582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071fa36-8fc9-47f8-86cf-0d0fe89f8156 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:21.833556 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071fa36-8fc9-47f8-86cf-0d0fe89f8156 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:21.838022 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1010/2039] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:21.838769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50319b31-5487-4587-ac5b-f69c659c230f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:21.848657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1030/2040] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:21.849626 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071fa36-8fc9-47f8-86cf-0d0fe89f8156 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:21.850307 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50319b31-5487-4587-ac5b-f69c659c230f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:21.857827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd290f8-2f8f-4156-82cc-3948ebf42998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:21.858461 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd290f8-2f8f-4156-82cc-3948ebf42998 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:21.858739 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd290f8-2f8f-4156-82cc-3948ebf42998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:21.858982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd290f8-2f8f-4156-82cc-3948ebf42998 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:21.859686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd290f8-2f8f-4156-82cc-3948ebf42998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:21.860913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071fa36-8fc9-47f8-86cf-0d0fe89f8156 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:21.861348 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd290f8-2f8f-4156-82cc-3948ebf42998 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:21.867329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291cf6ed-0a1c-4277-9597-bc87f6f5ea87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:21.869324 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cf6ed-0a1c-4277-9597-bc87f6f5ea87 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:21.869841 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cf6ed-0a1c-4277-9597-bc87f6f5ea87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:21.871021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cf6ed-0a1c-4277-9597-bc87f6f5ea87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:21.871926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291cf6ed-0a1c-4277-9597-bc87f6f5ea87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:21.874971 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291cf6ed-0a1c-4277-9597-bc87f6f5ea87 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:21.875099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1011/2041] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:21.875991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd290f8-2f8f-4156-82cc-3948ebf42998 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:21.887507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd290f8-2f8f-4156-82cc-3948ebf42998 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:21.893105 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fce03-34b8-46a5-916d-45a67eb87329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:21.893874 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fce03-34b8-46a5-916d-45a67eb87329 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:21.894156 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fce03-34b8-46a5-916d-45a67eb87329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:21.894436 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fce03-34b8-46a5-916d-45a67eb87329 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:21.895098 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875fce03-34b8-46a5-916d-45a67eb87329 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:21.898405 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875fce03-34b8-46a5-916d-45a67eb87329 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:21.912608 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1031/2042] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:21 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:21.913696 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291cf6ed-0a1c-4277-9597-bc87f6f5ea87 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:21.930834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1012/2043] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:21 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:21.931725 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fce03-34b8-46a5-916d-45a67eb87329 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:21.961992 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:21.991133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:22.019304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291cf6ed-0a1c-4277-9597-bc87f6f5ea87 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:22.038876 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fce03-34b8-46a5-916d-45a67eb87329 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:22.044599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd4d32-7beb-400e-883d-8d5ba9f4376b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:22.045338 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd4d32-7beb-400e-883d-8d5ba9f4376b admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:22.045602 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd4d32-7beb-400e-883d-8d5ba9f4376b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:22.045878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd4d32-7beb-400e-883d-8d5ba9f4376b admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:22.046631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd4d32-7beb-400e-883d-8d5ba9f4376b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:22.059814 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd4d32-7beb-400e-883d-8d5ba9f4376b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:22.103526 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1013/2044] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:21 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:22.104606 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd4d32-7beb-400e-883d-8d5ba9f4376b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:22.120411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd4d32-7beb-400e-883d-8d5ba9f4376b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:22.126070 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed861f9-5755-49e9-8b40-3e93e3bce0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:22.126323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed861f9-5755-49e9-8b40-3e93e3bce0d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:22.126605 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed861f9-5755-49e9-8b40-3e93e3bce0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:22.126877 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed861f9-5755-49e9-8b40-3e93e3bce0d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:22.127575 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed861f9-5755-49e9-8b40-3e93e3bce0d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:22.129545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed861f9-5755-49e9-8b40-3e93e3bce0d1 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:22.163728 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1014/2045] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:22 2022] POST /identity/v3/users => generated 284 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:22.163728 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed861f9-5755-49e9-8b40-3e93e3bce0d1 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:22.168892 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0560d2c-a4e0-4773-9f38-98eeb823ca12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:22.169555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0560d2c-a4e0-4773-9f38-98eeb823ca12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:22.169833 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0560d2c-a4e0-4773-9f38-98eeb823ca12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:22.170111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0560d2c-a4e0-4773-9f38-98eeb823ca12 admin admin] PATH_INFO: `/v3/projects/497a30a0ad36400b9cfd082b6aa0abc5/users/bf82d8704d004f25909fabc1b570146e/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:22.193392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed861f9-5755-49e9-8b40-3e93e3bce0d1 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:22.201055 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1caf499-b6f9-49fd-a561-dc86d27bc7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:22.202369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1caf499-b6f9-49fd-a561-dc86d27bc7fd admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:22.202987 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1caf499-b6f9-49fd-a561-dc86d27bc7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:22.203269 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1caf499-b6f9-49fd-a561-dc86d27bc7fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:22.203957 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1caf499-b6f9-49fd-a561-dc86d27bc7fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:22.205545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1caf499-b6f9-49fd-a561-dc86d27bc7fd admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:22.210789 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0560d2c-a4e0-4773-9f38-98eeb823ca12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=497a30a0ad36400b9cfd082b6aa0abc5, user_id=bf82d8704d004f25909fabc1b570146e, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:22.215132 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0560d2c-a4e0-4773-9f38-98eeb823ca12 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:22.217974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1015/2046] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:22.218909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1caf499-b6f9-49fd-a561-dc86d27bc7fd admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:22.231499 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1caf499-b6f9-49fd-a561-dc86d27bc7fd admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:22.237343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c5006-5ea5-496d-939e-24eacd5c941a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:22.237946 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c5006-5ea5-496d-939e-24eacd5c941a admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:22.238218 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c5006-5ea5-496d-939e-24eacd5c941a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:22.238512 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c5006-5ea5-496d-939e-24eacd5c941a admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:22.239152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c5006-5ea5-496d-939e-24eacd5c941a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:22.240776 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c5006-5ea5-496d-939e-24eacd5c941a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:22.251706 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1032/2047] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:21 2022] PUT /identity/v3/projects/497a30a0ad36400b9cfd082b6aa0abc5/users/bf82d8704d004f25909fabc1b570146e/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:22.251706 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0560d2c-a4e0-4773-9f38-98eeb823ca12 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:22.252100 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1016/2048] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:22.253021 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c5006-5ea5-496d-939e-24eacd5c941a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:22.260261 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f0560d2c-a4e0-4773-9f38-98eeb823ca12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:22.265808 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c5006-5ea5-496d-939e-24eacd5c941a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:22.327287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0d3cb-1a99-454a-a7af-ad32d58cb145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:22.327931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0d3cb-1a99-454a-a7af-ad32d58cb145 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:22.328216 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0d3cb-1a99-454a-a7af-ad32d58cb145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:22.328493 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0d3cb-1a99-454a-a7af-ad32d58cb145 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:22.329174 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f0d3cb-1a99-454a-a7af-ad32d58cb145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:22.330447 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0560d2c-a4e0-4773-9f38-98eeb823ca12 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:22.330798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f0d3cb-1a99-454a-a7af-ad32d58cb145 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:22.337273 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345288b-fc66-4978-9856-5ead7aae843b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:22.338730 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345288b-fc66-4978-9856-5ead7aae843b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:22.339012 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345288b-fc66-4978-9856-5ead7aae843b admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:22.339319 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345288b-fc66-4978-9856-5ead7aae843b admin admin] PATH_INFO: `/v3/projects/aa0a0b92bf1243c1821447758325bcf9` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:22.341597 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345288b-fc66-4978-9856-5ead7aae843b admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa0a0b92bf1243c1821447758325bcf9)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:22.342972 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1017/2049] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:22.344374 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00784f-ee46-47f6-b855-5b265f2fafc9 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:22.344665 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00784f-ee46-47f6-b855-5b265f2fafc9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:22.344945 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00784f-ee46-47f6-b855-5b265f2fafc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:22.345728 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345288b-fc66-4978-9856-5ead7aae843b admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:22.375347 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-fb00784f-ee46-47f6-b855-5b265f2fafc9 None None] MFA Rules not processed for user `bf82d8704d004f25909fabc1b570146e`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:22.376521 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-fb00784f-ee46-47f6-b855-5b265f2fafc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:22.433079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5345288b-fc66-4978-9856-5ead7aae843b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa0a0b92bf1243c1821447758325bcf9', 'request_id': 'req-5345288b-fc66-4978-9856-5ead7aae843b'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:22.444721 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5345288b-fc66-4978-9856-5ead7aae843b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa0a0b92bf1243c1821447758325bcf9', 'request_id': 'req-5345288b-fc66-4978-9856-5ead7aae843b'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:22.456133 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5345288b-fc66-4978-9856-5ead7aae843b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa0a0b92bf1243c1821447758325bcf9', 'request_id': 'req-5345288b-fc66-4978-9856-5ead7aae843b'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:22.462507 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:22.473046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1018/2050] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:22.475219 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00784f-ee46-47f6-b855-5b265f2fafc9 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:22.487750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5345288b-fc66-4978-9856-5ead7aae843b admin admin] The token cache is being invalidate because project aa0a0b92bf1243c1821447758325bcf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:22.488090 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5345288b-fc66-4978-9856-5ead7aae843b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:22.489268 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5345288b-fc66-4978-9856-5ead7aae843b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:22.490688 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1033/2051] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:22 2022] DELETE /identity/v3/projects/aa0a0b92bf1243c1821447758325bcf9 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:22.493371 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00784f-ee46-47f6-b855-5b265f2fafc9 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:22.495503 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345288b-fc66-4978-9856-5ead7aae843b admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:22.499273 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5345288b-fc66-4978-9856-5ead7aae843b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:22.501789 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:22.572311 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79554622-c847-4131-8586-8bb975c8d4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:22.572924 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79554622-c847-4131-8586-8bb975c8d4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:22.573203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79554622-c847-4131-8586-8bb975c8d4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:22.573478 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79554622-c847-4131-8586-8bb975c8d4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:22.574127 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79554622-c847-4131-8586-8bb975c8d4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:22.575734 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79554622-c847-4131-8586-8bb975c8d4b5 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:22.577516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345288b-fc66-4978-9856-5ead7aae843b admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:22.582139 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebb903-8e6f-45dd-96f8-e1973ed704cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:22.582789 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebb903-8e6f-45dd-96f8-e1973ed704cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:22.583066 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebb903-8e6f-45dd-96f8-e1973ed704cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:22.583337 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebb903-8e6f-45dd-96f8-e1973ed704cd admin admin] PATH_INFO: `/v3/projects/1128eb9736ff475cb7f0071b8971e5ed/users/365b2680c22e49f7a1fd09af955be555/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:22.585815 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1019/2052] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:22.586646 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79554622-c847-4131-8586-8bb975c8d4b5 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:22.588293 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-79554622-c847-4131-8586-8bb975c8d4b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:22.598572 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbebb903-8e6f-45dd-96f8-e1973ed704cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1128eb9736ff475cb7f0071b8971e5ed, user_id=365b2680c22e49f7a1fd09af955be555, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:22.601248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbebb903-8e6f-45dd-96f8-e1973ed704cd admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:22.615289 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1034/2053] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:22 2022] PUT /identity/v3/projects/1128eb9736ff475cb7f0071b8971e5ed/users/365b2680c22e49f7a1fd09af955be555/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:22.620661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebb903-8e6f-45dd-96f8-e1973ed704cd admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:22.622363 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-fbebb903-8e6f-45dd-96f8-e1973ed704cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:22.647811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79554622-c847-4131-8586-8bb975c8d4b5 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:22.690015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebb903-8e6f-45dd-96f8-e1973ed704cd admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:22.694735 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87191eb-1383-4911-bab0-52febd80c59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:22.695377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87191eb-1383-4911-bab0-52febd80c59a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:22.695681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87191eb-1383-4911-bab0-52febd80c59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:22.696090 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87191eb-1383-4911-bab0-52febd80c59a admin admin] PATH_INFO: `/v3/users/0fe036638eab4133b31bb30076261c17` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:22.697297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87191eb-1383-4911-bab0-52febd80c59a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fe036638eab4133b31bb30076261c17)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:22.698597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6eaed-107a-4a54-b9ab-4e2e0f782756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:22.699206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6eaed-107a-4a54-b9ab-4e2e0f782756 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:22.699314 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87191eb-1383-4911-bab0-52febd80c59a admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:22.699485 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6eaed-107a-4a54-b9ab-4e2e0f782756 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:22.699757 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6eaed-107a-4a54-b9ab-4e2e0f782756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:22.701556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-54c6eaed-107a-4a54-b9ab-4e2e0f782756 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:22.748909 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-f87191eb-1383-4911-bab0-52febd80c59a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fe036638eab4133b31bb30076261c17'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:22.749582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-f87191eb-1383-4911-bab0-52febd80c59a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fe036638eab4133b31bb30076261c17'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:22.750204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-f87191eb-1383-4911-bab0-52febd80c59a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fe036638eab4133b31bb30076261c17'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:22.757380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-f87191eb-1383-4911-bab0-52febd80c59a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fe036638eab4133b31bb30076261c17'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:22.763561 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6eaed-107a-4a54-b9ab-4e2e0f782756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:22.765037 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6eaed-107a-4a54-b9ab-4e2e0f782756 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:22.770384 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-f87191eb-1383-4911-bab0-52febd80c59a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fe036638eab4133b31bb30076261c17'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:22.773896 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1020/2054] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:22.774037 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:22.776649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6eaed-107a-4a54-b9ab-4e2e0f782756 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:22.778108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-54c6eaed-107a-4a54-b9ab-4e2e0f782756 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:22.788943 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1035/2055] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:22 2022] DELETE /identity/v3/users/0fe036638eab4133b31bb30076261c17 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:22.792740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e82b9f-0d45-4f0b-a62f-a1b3f4985038 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:22.792740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e82b9f-0d45-4f0b-a62f-a1b3f4985038 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:22.792740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e82b9f-0d45-4f0b-a62f-a1b3f4985038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:22.794034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6eaed-107a-4a54-b9ab-4e2e0f782756 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:22.806801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:22.872927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-02e82b9f-0d45-4f0b-a62f-a1b3f4985038 None None] MFA Rules not processed for user `365b2680c22e49f7a1fd09af955be555`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:22.874157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-02e82b9f-0d45-4f0b-a62f-a1b3f4985038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:22.926307 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103f7d3-ce30-4abe-b8b4-57564d274139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:22.926996 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103f7d3-ce30-4abe-b8b4-57564d274139 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:22.927273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103f7d3-ce30-4abe-b8b4-57564d274139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:22.927582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103f7d3-ce30-4abe-b8b4-57564d274139 admin admin] PATH_INFO: `/v3/projects/81453dbcae0b477bad4bf218f9392289/users/e63720466ea4459280de4ebf65a44847/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:22.959680 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103f7d3-ce30-4abe-b8b4-57564d274139 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81453dbcae0b477bad4bf218f9392289, user_id=e63720466ea4459280de4ebf65a44847, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:22.963580 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103f7d3-ce30-4abe-b8b4-57564d274139 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:22.987798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1021/2056] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:22 2022] PUT /identity/v3/projects/81453dbcae0b477bad4bf218f9392289/users/e63720466ea4459280de4ebf65a44847/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:22.993508 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b4ce7-9b9a-46a6-9969-226e5e60aa3c None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:22.994114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b4ce7-9b9a-46a6-9969-226e5e60aa3c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:22.994538 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b4ce7-9b9a-46a6-9969-226e5e60aa3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:23.053461 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-621b4ce7-9b9a-46a6-9969-226e5e60aa3c None None] MFA Rules not processed for user `e63720466ea4459280de4ebf65a44847`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:23.058937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-621b4ce7-9b9a-46a6-9969-226e5e60aa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:23.078595 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1036/2057] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:23.096394 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e82b9f-0d45-4f0b-a62f-a1b3f4985038 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:23.098336 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-02e82b9f-0d45-4f0b-a62f-a1b3f4985038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:23.226039 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e82b9f-0d45-4f0b-a62f-a1b3f4985038 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:23.226132 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1022/2058] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:23.227942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b4ce7-9b9a-46a6-9969-226e5e60aa3c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:23.232136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-621b4ce7-9b9a-46a6-9969-226e5e60aa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:23.359223 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2549041-a66a-40d2-babc-c17dfe2b4bad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:23.360012 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2549041-a66a-40d2-babc-c17dfe2b4bad service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:23.360369 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2549041-a66a-40d2-babc-c17dfe2b4bad service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:23.360583 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2549041-a66a-40d2-babc-c17dfe2b4bad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:23.381512 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b4ce7-9b9a-46a6-9969-226e5e60aa3c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:23.387039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e870a6b-b992-4451-b307-ff3b3919b854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:23.387820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e870a6b-b992-4451-b307-ff3b3919b854 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:23.388173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e870a6b-b992-4451-b307-ff3b3919b854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:23.388514 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e870a6b-b992-4451-b307-ff3b3919b854 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:23.389222 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e870a6b-b992-4451-b307-ff3b3919b854 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:23.401531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e870a6b-b992-4451-b307-ff3b3919b854 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:23.451467 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1023/2059] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:23 2022] POST /identity/v3/projects => generated 365 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:23.454426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e870a6b-b992-4451-b307-ff3b3919b854 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:23.457378 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6e870a6b-b992-4451-b307-ff3b3919b854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:23.462858 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:23.466239 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2549041-a66a-40d2-babc-c17dfe2b4bad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:23.468058 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2549041-a66a-40d2-babc-c17dfe2b4bad service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:23.481074 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1037/2060] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:23.482142 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2549041-a66a-40d2-babc-c17dfe2b4bad service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:23.484904 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:23.496912 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2549041-a66a-40d2-babc-c17dfe2b4bad service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:23.544516 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e870a6b-b992-4451-b307-ff3b3919b854 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:23.549960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ef2be-1b57-4dec-b2c8-8d92480f18aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:23.550381 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ef2be-1b57-4dec-b2c8-8d92480f18aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:23.550667 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ef2be-1b57-4dec-b2c8-8d92480f18aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:23.550940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ef2be-1b57-4dec-b2c8-8d92480f18aa admin admin] PATH_INFO: `/v3/projects/333b3a8c680f4e54a1f0946d4b011217` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:23.552116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ef2be-1b57-4dec-b2c8-8d92480f18aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=333b3a8c680f4e54a1f0946d4b011217)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:23.554015 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ef2be-1b57-4dec-b2c8-8d92480f18aa admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:23.575549 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8426086-334b-4683-9c89-68af71963d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:23.576415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8426086-334b-4683-9c89-68af71963d78 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:23.576787 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8426086-334b-4683-9c89-68af71963d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:23.577104 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8426086-334b-4683-9c89-68af71963d78 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:23.577814 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8426086-334b-4683-9c89-68af71963d78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:23.580329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8426086-334b-4683-9c89-68af71963d78 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:23.613030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1038/2061] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:23 2022] POST /identity/v3/users => generated 289 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:23.614186 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8426086-334b-4683-9c89-68af71963d78 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:23.620940 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c8426086-334b-4683-9c89-68af71963d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:23.627090 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ef1ef2be-1b57-4dec-b2c8-8d92480f18aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '333b3a8c680f4e54a1f0946d4b011217', 'request_id': 'req-ef1ef2be-1b57-4dec-b2c8-8d92480f18aa'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:23.636217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ef1ef2be-1b57-4dec-b2c8-8d92480f18aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '333b3a8c680f4e54a1f0946d4b011217', 'request_id': 'req-ef1ef2be-1b57-4dec-b2c8-8d92480f18aa'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:23.641222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8426086-334b-4683-9c89-68af71963d78 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:23.644623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ef1ef2be-1b57-4dec-b2c8-8d92480f18aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '333b3a8c680f4e54a1f0946d4b011217', 'request_id': 'req-ef1ef2be-1b57-4dec-b2c8-8d92480f18aa'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:23.677157 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ef1ef2be-1b57-4dec-b2c8-8d92480f18aa admin admin] The token cache is being invalidate because project 333b3a8c680f4e54a1f0946d4b011217 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:23.677654 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ef1ef2be-1b57-4dec-b2c8-8d92480f18aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:23.678350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ef1ef2be-1b57-4dec-b2c8-8d92480f18aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:23.680761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1024/2062] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:23 2022] DELETE /identity/v3/projects/333b3a8c680f4e54a1f0946d4b011217 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:23.681322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ef2be-1b57-4dec-b2c8-8d92480f18aa admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:23.682700 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ef1ef2be-1b57-4dec-b2c8-8d92480f18aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:23.725168 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef2a57-1822-4698-b866-46b304791736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:23.726818 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef2a57-1822-4698-b866-46b304791736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:23.728002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef2a57-1822-4698-b866-46b304791736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:23.728428 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef2a57-1822-4698-b866-46b304791736 admin admin] PATH_INFO: `/v3/users/365b2680c22e49f7a1fd09af955be555` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:23.729800 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef2a57-1822-4698-b866-46b304791736 admin admin] RBAC: Authorizing `identity:delete_user(user_id=365b2680c22e49f7a1fd09af955be555)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:23.733729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef2a57-1822-4698-b866-46b304791736 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:23.759110 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ef2be-1b57-4dec-b2c8-8d92480f18aa admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:23.764204 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ba4d9-df34-41e4-b140-7d1903ff50f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:23.764815 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ba4d9-df34-41e4-b140-7d1903ff50f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:23.765099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ba4d9-df34-41e4-b140-7d1903ff50f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:23.765380 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ba4d9-df34-41e4-b140-7d1903ff50f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:23.766198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ba4d9-df34-41e4-b140-7d1903ff50f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:23.767999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ba4d9-df34-41e4-b140-7d1903ff50f7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:23.779565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1025/2063] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:23.784145 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ba4d9-df34-41e4-b140-7d1903ff50f7 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:23.786312 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5c1ba4d9-df34-41e4-b140-7d1903ff50f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:23.802242 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ba4d9-df34-41e4-b140-7d1903ff50f7 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:23.808239 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5b782-f96d-4479-87da-dc00d806ef98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:23.809029 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5b782-f96d-4479-87da-dc00d806ef98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:23.809416 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5b782-f96d-4479-87da-dc00d806ef98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:23.809795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5b782-f96d-4479-87da-dc00d806ef98 admin admin] PATH_INFO: `/v3/users/9c9a37de5c6747329be790a758cbc283` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:23.811128 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5b782-f96d-4479-87da-dc00d806ef98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c9a37de5c6747329be790a758cbc283)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:23.813297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8bef2a57-1822-4698-b866-46b304791736 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '365b2680c22e49f7a1fd09af955be555'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:23.813498 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5b782-f96d-4479-87da-dc00d806ef98 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:23.813993 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8bef2a57-1822-4698-b866-46b304791736 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '365b2680c22e49f7a1fd09af955be555'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:23.814673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8bef2a57-1822-4698-b866-46b304791736 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '365b2680c22e49f7a1fd09af955be555'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:23.828985 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8bef2a57-1822-4698-b866-46b304791736 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '365b2680c22e49f7a1fd09af955be555'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:23.849122 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8bef2a57-1822-4698-b866-46b304791736 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '365b2680c22e49f7a1fd09af955be555'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:23.872632 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:23.873363 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1039/2064] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:23 2022] DELETE /identity/v3/users/365b2680c22e49f7a1fd09af955be555 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:23.874506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef2a57-1822-4698-b866-46b304791736 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:23.877720 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-8bef2a57-1822-4698-b866-46b304791736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:23.890010 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:23.895295 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ecc5b782-f96d-4479-87da-dc00d806ef98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c9a37de5c6747329be790a758cbc283'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:23.908391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ecc5b782-f96d-4479-87da-dc00d806ef98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c9a37de5c6747329be790a758cbc283'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:23.910421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ecc5b782-f96d-4479-87da-dc00d806ef98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c9a37de5c6747329be790a758cbc283'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:23.911151 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ecc5b782-f96d-4479-87da-dc00d806ef98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c9a37de5c6747329be790a758cbc283'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:23.931194 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ecc5b782-f96d-4479-87da-dc00d806ef98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c9a37de5c6747329be790a758cbc283'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:23.955840 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1026/2065] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:23 2022] DELETE /identity/v3/users/9c9a37de5c6747329be790a758cbc283 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:23.958588 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8de5d-76dc-40cb-b338-ab9d0f63d662 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:23.959854 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8de5d-76dc-40cb-b338-ab9d0f63d662 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:23.960609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8de5d-76dc-40cb-b338-ab9d0f63d662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:23.978871 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef2a57-1822-4698-b866-46b304791736 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:23.987678 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-04b8de5d-76dc-40cb-b338-ab9d0f63d662 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:23.989285 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-04b8de5d-76dc-40cb-b338-ab9d0f63d662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:24.070454 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b800b-ce65-413a-8b23-3af2205ca723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:24.071960 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b800b-ce65-413a-8b23-3af2205ca723 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:24.072243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b800b-ce65-413a-8b23-3af2205ca723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:24.072594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b800b-ce65-413a-8b23-3af2205ca723 admin admin] PATH_INFO: `/v3/projects/c31db90982dc439a91f48e7b8b413bfa/users/ea01aaf63999458691645e46634018c4/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:24.083917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1027/2066] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:24.093370 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b800b-ce65-413a-8b23-3af2205ca723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c31db90982dc439a91f48e7b8b413bfa, user_id=ea01aaf63999458691645e46634018c4, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:24.096163 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8de5d-76dc-40cb-b338-ab9d0f63d662 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:24.096627 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b800b-ce65-413a-8b23-3af2205ca723 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:24.099551 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-04b8de5d-76dc-40cb-b338-ab9d0f63d662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:24.117278 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1040/2067] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:23 2022] PUT /identity/v3/projects/c31db90982dc439a91f48e7b8b413bfa/users/ea01aaf63999458691645e46634018c4/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:24.122105 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b800b-ce65-413a-8b23-3af2205ca723 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:24.123920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e97b800b-ce65-413a-8b23-3af2205ca723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:24.179225 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8de5d-76dc-40cb-b338-ab9d0f63d662 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:24.201403 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b800b-ce65-413a-8b23-3af2205ca723 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:24.206281 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263cdbb-e3cd-4d2e-bba7-fa7a54d6db48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:24.206820 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263cdbb-e3cd-4d2e-bba7-fa7a54d6db48 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:24.207103 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263cdbb-e3cd-4d2e-bba7-fa7a54d6db48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:24.207376 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263cdbb-e3cd-4d2e-bba7-fa7a54d6db48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:24.208048 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263cdbb-e3cd-4d2e-bba7-fa7a54d6db48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:24.210224 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263cdbb-e3cd-4d2e-bba7-fa7a54d6db48 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:24.220876 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1041/2068] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:24:24.225002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263cdbb-e3cd-4d2e-bba7-fa7a54d6db48 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:24.236979 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263cdbb-e3cd-4d2e-bba7-fa7a54d6db48 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:24.237859 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8f39e-8b0f-4075-b6ae-d5f31aa9a376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:24.239386 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8f39e-8b0f-4075-b6ae-d5f31aa9a376 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:24.239966 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8f39e-8b0f-4075-b6ae-d5f31aa9a376 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:24.240570 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8f39e-8b0f-4075-b6ae-d5f31aa9a376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:24.243452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688d760-4cf9-4b4b-afc1-7b3a4b8ea654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:24.244143 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688d760-4cf9-4b4b-afc1-7b3a4b8ea654 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:24.244453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688d760-4cf9-4b4b-afc1-7b3a4b8ea654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:24.244729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688d760-4cf9-4b4b-afc1-7b3a4b8ea654 admin admin] PATH_INFO: `/v3/projects/c31db90982dc439a91f48e7b8b413bfa/users/ea01aaf63999458691645e46634018c4/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:24.246746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688d760-4cf9-4b4b-afc1-7b3a4b8ea654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c31db90982dc439a91f48e7b8b413bfa, user_id=ea01aaf63999458691645e46634018c4, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:24.249419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688d760-4cf9-4b4b-afc1-7b3a4b8ea654 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:24.252782 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c8f39e-8b0f-4075-b6ae-d5f31aa9a376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:24.254256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c8f39e-8b0f-4075-b6ae-d5f31aa9a376 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:24.261868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1042/2069] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:24 2022] PUT /identity/v3/projects/c31db90982dc439a91f48e7b8b413bfa/users/ea01aaf63999458691645e46634018c4/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:24.272563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582734ea-d189-41ad-87ef-b19999fe67e0 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:24.273033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582734ea-d189-41ad-87ef-b19999fe67e0 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:24.273523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582734ea-d189-41ad-87ef-b19999fe67e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:24.273853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1028/2070] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:24:24.307494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-582734ea-d189-41ad-87ef-b19999fe67e0 None None] MFA Rules not processed for user `ea01aaf63999458691645e46634018c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:24.308751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-582734ea-d189-41ad-87ef-b19999fe67e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:24.320231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:24.465980 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1043/2071] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:24.476658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8f39e-8b0f-4075-b6ae-d5f31aa9a376 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:24.479657 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-73c8f39e-8b0f-4075-b6ae-d5f31aa9a376 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:24.499482 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:24.596235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8f39e-8b0f-4075-b6ae-d5f31aa9a376 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:24.677320 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5e57b-c79e-433a-8b2e-d22fcefd4dc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:24.678541 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5e57b-c79e-433a-8b2e-d22fcefd4dc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:24.679259 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5e57b-c79e-433a-8b2e-d22fcefd4dc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:24.679538 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5e57b-c79e-433a-8b2e-d22fcefd4dc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:24.692892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a5e57b-c79e-433a-8b2e-d22fcefd4dc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:24.695195 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a5e57b-c79e-433a-8b2e-d22fcefd4dc1 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:24.718484 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1029/2072] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:24.814853 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582734ea-d189-41ad-87ef-b19999fe67e0 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:24.825046 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582734ea-d189-41ad-87ef-b19999fe67e0 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:24.829159 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2eef7-f05e-4861-8fcf-b0cd96395942 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:24.829757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2eef7-f05e-4861-8fcf-b0cd96395942 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:24.830037 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2eef7-f05e-4861-8fcf-b0cd96395942 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:24.830312 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2eef7-f05e-4861-8fcf-b0cd96395942 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:24.832117 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-4bc2eef7-f05e-4861-8fcf-b0cd96395942 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:24.896208 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2eef7-f05e-4861-8fcf-b0cd96395942 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:24.897653 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2eef7-f05e-4861-8fcf-b0cd96395942 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:24.906965 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1044/2073] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:25.126084 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5e57b-c79e-433a-8b2e-d22fcefd4dc1 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:25.127661 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-33a5e57b-c79e-433a-8b2e-d22fcefd4dc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:25.152470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2eef7-f05e-4861-8fcf-b0cd96395942 service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:25.154398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-4bc2eef7-f05e-4861-8fcf-b0cd96395942 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:25.292216 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5e57b-c79e-433a-8b2e-d22fcefd4dc1 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:25.304880 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7d70c-7f4b-416e-90ad-3d5c2971b64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:25.306077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7d70c-7f4b-416e-90ad-3d5c2971b64b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:25.306358 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7d70c-7f4b-416e-90ad-3d5c2971b64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:25.307165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7d70c-7f4b-416e-90ad-3d5c2971b64b admin admin] PATH_INFO: `/v3/projects/1bf27031fcec48d69fbb0281c8d7c8c7` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:25.307823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7d70c-7f4b-416e-90ad-3d5c2971b64b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bf27031fcec48d69fbb0281c8d7c8c7)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:25.309760 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7d70c-7f4b-416e-90ad-3d5c2971b64b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:25.312419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2eef7-f05e-4861-8fcf-b0cd96395942 service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:25.319381 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bd82d-fccd-4cad-9e03-a1a03e6b8f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:25.320273 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bd82d-fccd-4cad-9e03-a1a03e6b8f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:25.321223 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bd82d-fccd-4cad-9e03-a1a03e6b8f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:25.321587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bd82d-fccd-4cad-9e03-a1a03e6b8f52 admin admin] PATH_INFO: `/v3/projects/1128eb9736ff475cb7f0071b8971e5ed` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:25.322933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7bd82d-fccd-4cad-9e03-a1a03e6b8f52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1128eb9736ff475cb7f0071b8971e5ed)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:25.339831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7bd82d-fccd-4cad-9e03-a1a03e6b8f52 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:25.418812 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-07a7d70c-7f4b-416e-90ad-3d5c2971b64b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bf27031fcec48d69fbb0281c8d7c8c7', 'request_id': 'req-07a7d70c-7f4b-416e-90ad-3d5c2971b64b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:25.429569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-07a7d70c-7f4b-416e-90ad-3d5c2971b64b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bf27031fcec48d69fbb0281c8d7c8c7', 'request_id': 'req-07a7d70c-7f4b-416e-90ad-3d5c2971b64b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:25.434689 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:25.444214 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-07a7d70c-7f4b-416e-90ad-3d5c2971b64b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bf27031fcec48d69fbb0281c8d7c8c7', 'request_id': 'req-07a7d70c-7f4b-416e-90ad-3d5c2971b64b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:25.444452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4e7bd82d-fccd-4cad-9e03-a1a03e6b8f52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1128eb9736ff475cb7f0071b8971e5ed', 'request_id': 'req-4e7bd82d-fccd-4cad-9e03-a1a03e6b8f52'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:25.457560 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4e7bd82d-fccd-4cad-9e03-a1a03e6b8f52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1128eb9736ff475cb7f0071b8971e5ed', 'request_id': 'req-4e7bd82d-fccd-4cad-9e03-a1a03e6b8f52'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:25.460927 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:25.468519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4e7bd82d-fccd-4cad-9e03-a1a03e6b8f52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1128eb9736ff475cb7f0071b8971e5ed', 'request_id': 'req-4e7bd82d-fccd-4cad-9e03-a1a03e6b8f52'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:25.471587 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-07a7d70c-7f4b-416e-90ad-3d5c2971b64b admin admin] The token cache is being invalidate because project 1bf27031fcec48d69fbb0281c8d7c8c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:25.472082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-07a7d70c-7f4b-416e-90ad-3d5c2971b64b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:25.473660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-07a7d70c-7f4b-416e-90ad-3d5c2971b64b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:25.475685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1030/2074] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:25 2022] DELETE /identity/v3/projects/1bf27031fcec48d69fbb0281c8d7c8c7 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:25.482402 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bb17c-b262-49cd-8433-d29963ee338c None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:25.483893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bb17c-b262-49cd-8433-d29963ee338c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:25.484191 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bb17c-b262-49cd-8433-d29963ee338c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:25.487757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4e7bd82d-fccd-4cad-9e03-a1a03e6b8f52 admin admin] The token cache is being invalidate because project 1128eb9736ff475cb7f0071b8971e5ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:25.488127 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4e7bd82d-fccd-4cad-9e03-a1a03e6b8f52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:25.488777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4e7bd82d-fccd-4cad-9e03-a1a03e6b8f52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:25.490438 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1045/2075] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:25 2022] DELETE /identity/v3/projects/1128eb9736ff475cb7f0071b8971e5ed => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:25.502628 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ceede-c235-4ddf-bc1a-3a26afbc0f42 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:25.502999 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ceede-c235-4ddf-bc1a-3a26afbc0f42 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:25.503337 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ceede-c235-4ddf-bc1a-3a26afbc0f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:25.517109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-b17bb17c-b262-49cd-8433-d29963ee338c None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:25.518113 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b17bb17c-b262-49cd-8433-d29963ee338c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:25.537390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-539ceede-c235-4ddf-bc1a-3a26afbc0f42 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:25.540183 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-539ceede-c235-4ddf-bc1a-3a26afbc0f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:25.605327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1031/2076] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:25.615339 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe002267-ca2b-41f0-830f-cc07b0144449 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:25.615632 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe002267-ca2b-41f0-830f-cc07b0144449 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:25.615960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe002267-ca2b-41f0-830f-cc07b0144449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:25.631231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1046/2077] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:25.640875 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-fe002267-ca2b-41f0-830f-cc07b0144449 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:25.641290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ffcae-1c3f-40c7-9304-d1119371ea2f None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:25.641680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ffcae-1c3f-40c7-9304-d1119371ea2f None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:25.642058 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ffcae-1c3f-40c7-9304-d1119371ea2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:25.642221 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-fe002267-ca2b-41f0-830f-cc07b0144449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:25.655068 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1032/2078] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:25.656095 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe002267-ca2b-41f0-830f-cc07b0144449 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:25.667579 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe002267-ca2b-41f0-830f-cc07b0144449 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:25.681428 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-dc9ffcae-1c3f-40c7-9304-d1119371ea2f None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:25.682956 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-dc9ffcae-1c3f-40c7-9304-d1119371ea2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:25.684952 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8ea1f-8646-4471-a064-800061cd25e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:25.686308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8ea1f-8646-4471-a064-800061cd25e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:25.686602 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8ea1f-8646-4471-a064-800061cd25e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:25.686882 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8ea1f-8646-4471-a064-800061cd25e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:25.687901 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8ea1f-8646-4471-a064-800061cd25e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:25.688233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8ea1f-8646-4471-a064-800061cd25e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:25.690673 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8ea1f-8646-4471-a064-800061cd25e4 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:25.693282 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1047/2079] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:24:25.706855 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ffcae-1c3f-40c7-9304-d1119371ea2f None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:25.713735 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1033/2080] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:25.718370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8ea1f-8646-4471-a064-800061cd25e4 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:25.719381 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ffcae-1c3f-40c7-9304-d1119371ea2f None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:25.725634 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e6ce1-c27a-49ee-b47b-be55f379ca54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:25.726181 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e6ce1-c27a-49ee-b47b-be55f379ca54 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:25.726455 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e6ce1-c27a-49ee-b47b-be55f379ca54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:25.726724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e6ce1-c27a-49ee-b47b-be55f379ca54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:25.727990 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e6ce1-c27a-49ee-b47b-be55f379ca54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:25.728292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e6ce1-c27a-49ee-b47b-be55f379ca54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:25.729889 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e6ce1-c27a-49ee-b47b-be55f379ca54 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:25.732949 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8ea1f-8646-4471-a064-800061cd25e4 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:25.743921 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8b550-915a-47b2-88d8-89a9f97dae27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:25.744747 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8b550-915a-47b2-88d8-89a9f97dae27 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:25.745108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8b550-915a-47b2-88d8-89a9f97dae27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:25.748589 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8b550-915a-47b2-88d8-89a9f97dae27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:25.748713 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1048/2081] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:24:25.748793 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec8b550-915a-47b2-88d8-89a9f97dae27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:25.748793 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec8b550-915a-47b2-88d8-89a9f97dae27 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:25.752906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e6ce1-c27a-49ee-b47b-be55f379ca54 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:25.780334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e6ce1-c27a-49ee-b47b-be55f379ca54 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:25.786550 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb21b8f-6931-4cb3-9357-d39450dc1348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:25.787373 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb21b8f-6931-4cb3-9357-d39450dc1348 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:25.787865 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb21b8f-6931-4cb3-9357-d39450dc1348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:25.788322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb21b8f-6931-4cb3-9357-d39450dc1348 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:25.789158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb21b8f-6931-4cb3-9357-d39450dc1348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:25.791473 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb21b8f-6931-4cb3-9357-d39450dc1348 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:25.814322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1034/2082] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:25 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:25.818998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8b550-915a-47b2-88d8-89a9f97dae27 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:25.833342 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1049/2083] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:25 2022] POST /identity/v3/projects => generated 375 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:25.834820 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb21b8f-6931-4cb3-9357-d39450dc1348 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:25.910095 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8b550-915a-47b2-88d8-89a9f97dae27 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:25.916698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abed14-fa3c-42a2-a114-f40b130421dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:25.917992 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abed14-fa3c-42a2-a114-f40b130421dd admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:25.918794 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abed14-fa3c-42a2-a114-f40b130421dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:25.919170 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abed14-fa3c-42a2-a114-f40b130421dd admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:25.919634 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb21b8f-6931-4cb3-9357-d39450dc1348 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:25.920300 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9abed14-fa3c-42a2-a114-f40b130421dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:25.922518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9abed14-fa3c-42a2-a114-f40b130421dd admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:25.926487 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b2167-ef8e-42db-87d6-50de7baafa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:25.927103 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b2167-ef8e-42db-87d6-50de7baafa89 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:25.927387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b2167-ef8e-42db-87d6-50de7baafa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:25.927840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b2167-ef8e-42db-87d6-50de7baafa89 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:25.928484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b2167-ef8e-42db-87d6-50de7baafa89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:25.931722 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b2167-ef8e-42db-87d6-50de7baafa89 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:25.952229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1035/2084] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:25 2022] POST /identity/v3/users => generated 289 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:24:25.956554 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abed14-fa3c-42a2-a114-f40b130421dd admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:25.966649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1050/2085] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:25 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:25.970773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b2167-ef8e-42db-87d6-50de7baafa89 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:25.973487 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abed14-fa3c-42a2-a114-f40b130421dd admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:25.982991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f474c8-50c4-483e-827b-ef54a5a9c4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:25.983890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f474c8-50c4-483e-827b-ef54a5a9c4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:25.984269 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f474c8-50c4-483e-827b-ef54a5a9c4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:25.985351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f474c8-50c4-483e-827b-ef54a5a9c4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:25.985780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b2167-ef8e-42db-87d6-50de7baafa89 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:25.986565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f474c8-50c4-483e-827b-ef54a5a9c4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:25.989068 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f474c8-50c4-483e-827b-ef54a5a9c4b5 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:25.991197 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989d617-1837-4df1-989f-a692e5d5be5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:25.992107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989d617-1837-4df1-989f-a692e5d5be5a admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:25.993290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989d617-1837-4df1-989f-a692e5d5be5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:25.993716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989d617-1837-4df1-989f-a692e5d5be5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:25.994561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989d617-1837-4df1-989f-a692e5d5be5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:25.996535 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989d617-1837-4df1-989f-a692e5d5be5a admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:26.004483 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1036/2086] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:26.011800 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f474c8-50c4-483e-827b-ef54a5a9c4b5 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:26.011958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1051/2087] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:24:26.014891 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989d617-1837-4df1-989f-a692e5d5be5a admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:26.026280 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f474c8-50c4-483e-827b-ef54a5a9c4b5 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:26.026619 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989d617-1837-4df1-989f-a692e5d5be5a admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:26.031505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df7040-a3ee-48ca-882f-e126c10ee616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:26.032167 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df7040-a3ee-48ca-882f-e126c10ee616 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:26.032493 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df7040-a3ee-48ca-882f-e126c10ee616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:26.032781 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df7040-a3ee-48ca-882f-e126c10ee616 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:26.033428 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df7040-a3ee-48ca-882f-e126c10ee616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:26.035032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df7040-a3ee-48ca-882f-e126c10ee616 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:26.037402 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce965b-f1e8-4685-8f66-634ff62669d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:26.037974 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:26.038083 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce965b-f1e8-4685-8f66-634ff62669d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:26.038286 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce965b-f1e8-4685-8f66-634ff62669d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:26.038563 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce965b-f1e8-4685-8f66-634ff62669d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:26.039220 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce965b-f1e8-4685-8f66-634ff62669d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:26.041253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce965b-f1e8-4685-8f66-634ff62669d6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:26.055617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1037/2088] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:24:26.056530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1052/2089] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:24:26.057293 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce965b-f1e8-4685-8f66-634ff62669d6 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:26.060056 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df7040-a3ee-48ca-882f-e126c10ee616 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:26.069544 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce965b-f1e8-4685-8f66-634ff62669d6 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:26.072006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df7040-a3ee-48ca-882f-e126c10ee616 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:26.075465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cff712-38b6-4dd5-94a4-c86e12fdc08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:26.076273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cff712-38b6-4dd5-94a4-c86e12fdc08b admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:26.076556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cff712-38b6-4dd5-94a4-c86e12fdc08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:26.076830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cff712-38b6-4dd5-94a4-c86e12fdc08b admin admin] PATH_INFO: `/v3/projects/6c14c7e4417f4da0b4edc45ca7bcb176/users/aa5239f9f3024874847b609ea8f98575/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:26.077959 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e8f25-b1b5-43fa-8b41-bac69fba9e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:26.078822 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e8f25-b1b5-43fa-8b41-bac69fba9e60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:26.079561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e8f25-b1b5-43fa-8b41-bac69fba9e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:26.080950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e8f25-b1b5-43fa-8b41-bac69fba9e60 admin admin] PATH_INFO: `/v3/projects/80e8b3a36fc641d5b0eb5968394a108d/users/3dd26ea6bdee48a8ab19dbd157083b5b/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:26.098006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cff712-38b6-4dd5-94a4-c86e12fdc08b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c14c7e4417f4da0b4edc45ca7bcb176, user_id=aa5239f9f3024874847b609ea8f98575, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:26.100552 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e8f25-b1b5-43fa-8b41-bac69fba9e60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e8b3a36fc641d5b0eb5968394a108d, user_id=3dd26ea6bdee48a8ab19dbd157083b5b, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:26.101629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cff712-38b6-4dd5-94a4-c86e12fdc08b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:26.103665 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e8f25-b1b5-43fa-8b41-bac69fba9e60 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:26.126252 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1053/2091] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:26 2022] PUT /identity/v3/projects/80e8b3a36fc641d5b0eb5968394a108d/users/3dd26ea6bdee48a8ab19dbd157083b5b/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:26.126483 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1038/2091] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:26 2022] PUT /identity/v3/projects/6c14c7e4417f4da0b4edc45ca7bcb176/users/aa5239f9f3024874847b609ea8f98575/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:26.135587 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1f671-4e12-477f-8f6b-a942a10ec750 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:26.135902 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1f671-4e12-477f-8f6b-a942a10ec750 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:26.136173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1f671-4e12-477f-8f6b-a942a10ec750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:26.139636 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a0859-f128-4451-a6e5-e3c9192e8b80 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:26.140248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a0859-f128-4451-a6e5-e3c9192e8b80 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:26.140677 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a0859-f128-4451-a6e5-e3c9192e8b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:26.144114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:26.170519 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f1b1f671-4e12-477f-8f6b-a942a10ec750 None None] MFA Rules not processed for user `aa5239f9f3024874847b609ea8f98575`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:26.171892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f1b1f671-4e12-477f-8f6b-a942a10ec750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:26.178653 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-6a1a0859-f128-4451-a6e5-e3c9192e8b80 None None] MFA Rules not processed for user `3dd26ea6bdee48a8ab19dbd157083b5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:26.180823 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-6a1a0859-f128-4451-a6e5-e3c9192e8b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:26.253655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1039/2092] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:24:26.265005 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1f671-4e12-477f-8f6b-a942a10ec750 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:26.266948 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f1b1f671-4e12-477f-8f6b-a942a10ec750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:26.272658 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1054/2093] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:24:26.285082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a0859-f128-4451-a6e5-e3c9192e8b80 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:26.287836 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-6a1a0859-f128-4451-a6e5-e3c9192e8b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:26.377531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1f671-4e12-477f-8f6b-a942a10ec750 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:26.387499 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46898a2e-9901-432d-9f1c-59d23240c4f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:26.388459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46898a2e-9901-432d-9f1c-59d23240c4f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:26.388743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46898a2e-9901-432d-9f1c-59d23240c4f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:26.389015 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46898a2e-9901-432d-9f1c-59d23240c4f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:26.390393 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a0859-f128-4451-a6e5-e3c9192e8b80 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:26.395716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf3794-18ba-4f7a-a839-0d8d3a392e60 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88670b0e1a0f460d8f74917c92db9b9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:26.396563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf3794-18ba-4f7a-a839-0d8d3a392e60 service glance] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:26.397155 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf3794-18ba-4f7a-a839-0d8d3a392e60 service glance] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:26.397600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf3794-18ba-4f7a-a839-0d8d3a392e60 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:26.402135 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46898a2e-9901-432d-9f1c-59d23240c4f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:26.404052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46898a2e-9901-432d-9f1c-59d23240c4f1 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:26.412160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf3794-18ba-4f7a-a839-0d8d3a392e60 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:26.415311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf3794-18ba-4f7a-a839-0d8d3a392e60 service glance] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:26.425884 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1040/2094] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:26.431475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1055/2095] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:24:34.367363 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46898a2e-9901-432d-9f1c-59d23240c4f1 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:34.373983 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-46898a2e-9901-432d-9f1c-59d23240c4f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:34.455025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46898a2e-9901-432d-9f1c-59d23240c4f1 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:34.460887 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdcdb9-9a9b-4de2-8ca3-6b5f36a44b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:34.461666 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdcdb9-9a9b-4de2-8ca3-6b5f36a44b62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:34.462079 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdcdb9-9a9b-4de2-8ca3-6b5f36a44b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:34.462457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdcdb9-9a9b-4de2-8ca3-6b5f36a44b62 admin admin] PATH_INFO: `/v3/users/e63720466ea4459280de4ebf65a44847` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:34.463937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fdcdb9-9a9b-4de2-8ca3-6b5f36a44b62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e63720466ea4459280de4ebf65a44847)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:34.465974 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fdcdb9-9a9b-4de2-8ca3-6b5f36a44b62 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:34.531939 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:34.545058 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-65fdcdb9-9a9b-4de2-8ca3-6b5f36a44b62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e63720466ea4459280de4ebf65a44847'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:34.557529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-65fdcdb9-9a9b-4de2-8ca3-6b5f36a44b62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e63720466ea4459280de4ebf65a44847'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:34.558173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-65fdcdb9-9a9b-4de2-8ca3-6b5f36a44b62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e63720466ea4459280de4ebf65a44847'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:34.558843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-65fdcdb9-9a9b-4de2-8ca3-6b5f36a44b62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e63720466ea4459280de4ebf65a44847'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:34.574227 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-65fdcdb9-9a9b-4de2-8ca3-6b5f36a44b62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e63720466ea4459280de4ebf65a44847'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:34.597571 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1041/2096] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:34 2022] DELETE /identity/v3/users/e63720466ea4459280de4ebf65a44847 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:34.602906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5b006-2133-4c08-8ff7-8c675e60543c None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:34.603425 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5b006-2133-4c08-8ff7-8c675e60543c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:34.603504 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5b006-2133-4c08-8ff7-8c675e60543c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:34.623918 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-77e5b006-2133-4c08-8ff7-8c675e60543c None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:34.625151 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-77e5b006-2133-4c08-8ff7-8c675e60543c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:34.674527 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:34.694375 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1056/2097] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:24:34.704272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdcdb9-9a9b-4de2-8ca3-6b5f36a44b62 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:34.705722 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-65fdcdb9-9a9b-4de2-8ca3-6b5f36a44b62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:34.767582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdcdb9-9a9b-4de2-8ca3-6b5f36a44b62 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:34.771197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3846d78-5e19-49c0-bfe9-d2df0e5ffbe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:34.771833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3846d78-5e19-49c0-bfe9-d2df0e5ffbe1 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:34.772110 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3846d78-5e19-49c0-bfe9-d2df0e5ffbe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:34.772379 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3846d78-5e19-49c0-bfe9-d2df0e5ffbe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:34.781465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3846d78-5e19-49c0-bfe9-d2df0e5ffbe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:34.782875 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3846d78-5e19-49c0-bfe9-d2df0e5ffbe1 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:34.792596 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1042/2098] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:35.063517 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5b006-2133-4c08-8ff7-8c675e60543c None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:35.065056 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-77e5b006-2133-4c08-8ff7-8c675e60543c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:35.079025 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5b006-2133-4c08-8ff7-8c675e60543c None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:35.084174 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8706ea-173b-428d-b6b0-e9297e8020bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:35.084816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8706ea-173b-428d-b6b0-e9297e8020bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:35.085090 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8706ea-173b-428d-b6b0-e9297e8020bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:35.085419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8706ea-173b-428d-b6b0-e9297e8020bc admin admin] PATH_INFO: `/v3/projects/81453dbcae0b477bad4bf218f9392289` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:35.086544 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8706ea-173b-428d-b6b0-e9297e8020bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=81453dbcae0b477bad4bf218f9392289)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:35.088954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8706ea-173b-428d-b6b0-e9297e8020bc admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:35.134849 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5f8706ea-173b-428d-b6b0-e9297e8020bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81453dbcae0b477bad4bf218f9392289', 'request_id': 'req-5f8706ea-173b-428d-b6b0-e9297e8020bc'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:35.141072 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5f8706ea-173b-428d-b6b0-e9297e8020bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81453dbcae0b477bad4bf218f9392289', 'request_id': 'req-5f8706ea-173b-428d-b6b0-e9297e8020bc'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:35.147095 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5f8706ea-173b-428d-b6b0-e9297e8020bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81453dbcae0b477bad4bf218f9392289', 'request_id': 'req-5f8706ea-173b-428d-b6b0-e9297e8020bc'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:35.161841 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5f8706ea-173b-428d-b6b0-e9297e8020bc admin admin] The token cache is being invalidate because project 81453dbcae0b477bad4bf218f9392289 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:35.162271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5f8706ea-173b-428d-b6b0-e9297e8020bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:35.162905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5f8706ea-173b-428d-b6b0-e9297e8020bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:35.164428 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1057/2099] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:35 2022] DELETE /identity/v3/projects/81453dbcae0b477bad4bf218f9392289 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:35.169008 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3846d78-5e19-49c0-bfe9-d2df0e5ffbe1 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:35.178647 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3846d78-5e19-49c0-bfe9-d2df0e5ffbe1 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:35.235779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4359402-882e-4ee3-b3a4-2c84ab3224ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:35.236491 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4359402-882e-4ee3-b3a4-2c84ab3224ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:35.236806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4359402-882e-4ee3-b3a4-2c84ab3224ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:35.237087 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4359402-882e-4ee3-b3a4-2c84ab3224ca admin admin] PATH_INFO: `/v3/users/ea01aaf63999458691645e46634018c4` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:35.238246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4359402-882e-4ee3-b3a4-2c84ab3224ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea01aaf63999458691645e46634018c4)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:35.240248 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4359402-882e-4ee3-b3a4-2c84ab3224ca admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:35.289307 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d4359402-882e-4ee3-b3a4-2c84ab3224ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea01aaf63999458691645e46634018c4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:35.297601 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d4359402-882e-4ee3-b3a4-2c84ab3224ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea01aaf63999458691645e46634018c4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:35.298214 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d4359402-882e-4ee3-b3a4-2c84ab3224ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea01aaf63999458691645e46634018c4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:35.298815 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d4359402-882e-4ee3-b3a4-2c84ab3224ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea01aaf63999458691645e46634018c4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:35.303228 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:35.316178 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d4359402-882e-4ee3-b3a4-2c84ab3224ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea01aaf63999458691645e46634018c4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:35.330714 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1043/2100] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:35 2022] DELETE /identity/v3/users/ea01aaf63999458691645e46634018c4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:35.806854 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8706ea-173b-428d-b6b0-e9297e8020bc admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:35.808399 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5f8706ea-173b-428d-b6b0-e9297e8020bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:35.825202 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8706ea-173b-428d-b6b0-e9297e8020bc admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:35.891479 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef37eae-7156-48b1-b544-067ced0e9c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:35.895848 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef37eae-7156-48b1-b544-067ced0e9c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:35.896140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef37eae-7156-48b1-b544-067ced0e9c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:35.896422 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef37eae-7156-48b1-b544-067ced0e9c54 admin admin] PATH_INFO: `/v3/projects/c31db90982dc439a91f48e7b8b413bfa` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:35.897665 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef37eae-7156-48b1-b544-067ced0e9c54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c31db90982dc439a91f48e7b8b413bfa)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:35.899600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef37eae-7156-48b1-b544-067ced0e9c54 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:35.949437 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-2ef37eae-7156-48b1-b544-067ced0e9c54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c31db90982dc439a91f48e7b8b413bfa', 'request_id': 'req-2ef37eae-7156-48b1-b544-067ced0e9c54'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:35.955568 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-2ef37eae-7156-48b1-b544-067ced0e9c54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c31db90982dc439a91f48e7b8b413bfa', 'request_id': 'req-2ef37eae-7156-48b1-b544-067ced0e9c54'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:35.961712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-2ef37eae-7156-48b1-b544-067ced0e9c54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c31db90982dc439a91f48e7b8b413bfa', 'request_id': 'req-2ef37eae-7156-48b1-b544-067ced0e9c54'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:35.976730 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:35.981714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-2ef37eae-7156-48b1-b544-067ced0e9c54 admin admin] The token cache is being invalidate because project c31db90982dc439a91f48e7b8b413bfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:35.982004 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-2ef37eae-7156-48b1-b544-067ced0e9c54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:35.982670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-2ef37eae-7156-48b1-b544-067ced0e9c54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:35.987620 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1058/2101] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:35 2022] DELETE /identity/v3/projects/c31db90982dc439a91f48e7b8b413bfa => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:35.989299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637be13e-8dce-4f25-9489-7be863ecec71 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:35.989762 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637be13e-8dce-4f25-9489-7be863ecec71 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:35.990091 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637be13e-8dce-4f25-9489-7be863ecec71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:36.011460 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-637be13e-8dce-4f25-9489-7be863ecec71 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:36.012717 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-637be13e-8dce-4f25-9489-7be863ecec71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:36.075167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1044/2102] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:36.084138 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b6834-45ac-4439-ba9d-d02c85645866 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:36.084696 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b6834-45ac-4439-ba9d-d02c85645866 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:36.085057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b6834-45ac-4439-ba9d-d02c85645866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:36.105462 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-eb4b6834-45ac-4439-ba9d-d02c85645866 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:36.106695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-eb4b6834-45ac-4439-ba9d-d02c85645866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:36.114539 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1059/2103] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:36.119400 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637be13e-8dce-4f25-9489-7be863ecec71 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:36.129060 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637be13e-8dce-4f25-9489-7be863ecec71 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:36.133607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01472c-07bf-411b-84b4-1b49db6d1208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:36.134191 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01472c-07bf-411b-84b4-1b49db6d1208 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:36.134466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01472c-07bf-411b-84b4-1b49db6d1208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:36.134736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01472c-07bf-411b-84b4-1b49db6d1208 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:36.135369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db01472c-07bf-411b-84b4-1b49db6d1208 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:36.135668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db01472c-07bf-411b-84b4-1b49db6d1208 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:36.137291 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db01472c-07bf-411b-84b4-1b49db6d1208 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:36.148797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1045/2104] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:36.152588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b6834-45ac-4439-ba9d-d02c85645866 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:36.161709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b6834-45ac-4439-ba9d-d02c85645866 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:36.167312 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65fde3-fda3-43f4-a1c6-49a6466c0c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:36.167934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65fde3-fda3-43f4-a1c6-49a6466c0c6c admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:36.168212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65fde3-fda3-43f4-a1c6-49a6466c0c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:36.168480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65fde3-fda3-43f4-a1c6-49a6466c0c6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:36.169094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65fde3-fda3-43f4-a1c6-49a6466c0c6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:36.171301 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65fde3-fda3-43f4-a1c6-49a6466c0c6c admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:36.201229 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1060/2105] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:36 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:36.205572 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01472c-07bf-411b-84b4-1b49db6d1208 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:36.218011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01472c-07bf-411b-84b4-1b49db6d1208 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:36.280668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4db529-f1fd-441e-9247-5dde1f08afc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:36.281274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4db529-f1fd-441e-9247-5dde1f08afc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:36.281555 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4db529-f1fd-441e-9247-5dde1f08afc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:36.281830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4db529-f1fd-441e-9247-5dde1f08afc2 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:36.282475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4db529-f1fd-441e-9247-5dde1f08afc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:36.284484 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4db529-f1fd-441e-9247-5dde1f08afc2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:36.311545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1046/2106] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:36 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:36.312803 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65fde3-fda3-43f4-a1c6-49a6466c0c6c admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:36.323553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65fde3-fda3-43f4-a1c6-49a6466c0c6c admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:36.328332 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e29cb-e840-4479-ae01-8b51f7b1acd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:36.328866 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e29cb-e840-4479-ae01-8b51f7b1acd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:36.329147 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e29cb-e840-4479-ae01-8b51f7b1acd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:36.329420 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e29cb-e840-4479-ae01-8b51f7b1acd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:36.330062 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e29cb-e840-4479-ae01-8b51f7b1acd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:36.331644 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e29cb-e840-4479-ae01-8b51f7b1acd2 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:36.342318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1061/2107] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:36.346674 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4db529-f1fd-441e-9247-5dde1f08afc2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:36.356991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4db529-f1fd-441e-9247-5dde1f08afc2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:36.361953 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a13714-4af7-448c-b18c-0a7fde13c106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:36.362542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a13714-4af7-448c-b18c-0a7fde13c106 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:36.362820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a13714-4af7-448c-b18c-0a7fde13c106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:36.363089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a13714-4af7-448c-b18c-0a7fde13c106 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:36.363724 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a13714-4af7-448c-b18c-0a7fde13c106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:36.365768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a13714-4af7-448c-b18c-0a7fde13c106 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:36.378142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1047/2108] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:36.382642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e29cb-e840-4479-ae01-8b51f7b1acd2 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:36.393392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e29cb-e840-4479-ae01-8b51f7b1acd2 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:36.398172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68121bff-75a1-4049-81ea-64f814130365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:36.398810 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68121bff-75a1-4049-81ea-64f814130365 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:36.399122 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68121bff-75a1-4049-81ea-64f814130365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:36.400069 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68121bff-75a1-4049-81ea-64f814130365 admin admin] PATH_INFO: `/v3/projects/f6f2757d16ea4e74b41d2c8ba52aa32a/users/03bbd4546bba43e5a75b4ad669178237/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:36.414457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68121bff-75a1-4049-81ea-64f814130365 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6f2757d16ea4e74b41d2c8ba52aa32a, user_id=03bbd4546bba43e5a75b4ad669178237, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:36.417305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68121bff-75a1-4049-81ea-64f814130365 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:36.430037 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1062/2109] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:36 2022] PUT /identity/v3/projects/f6f2757d16ea4e74b41d2c8ba52aa32a/users/03bbd4546bba43e5a75b4ad669178237/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:36.440686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ae367-e3ea-46c7-85a9-1a70d8f7c449 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:36.440971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ae367-e3ea-46c7-85a9-1a70d8f7c449 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:36.441245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ae367-e3ea-46c7-85a9-1a70d8f7c449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:36.471131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-862ae367-e3ea-46c7-85a9-1a70d8f7c449 None None] MFA Rules not processed for user `03bbd4546bba43e5a75b4ad669178237`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:36.472377 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-862ae367-e3ea-46c7-85a9-1a70d8f7c449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:36.502178 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:36.541855 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1048/2110] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:36.552984 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68121bff-75a1-4049-81ea-64f814130365 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:36.554734 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-68121bff-75a1-4049-81ea-64f814130365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:36.612627 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68121bff-75a1-4049-81ea-64f814130365 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:36.618822 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c3212-41d3-487b-afee-bffe2fb521d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:36.619715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c3212-41d3-487b-afee-bffe2fb521d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:36.620230 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c3212-41d3-487b-afee-bffe2fb521d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:36.620752 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c3212-41d3-487b-afee-bffe2fb521d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:36.630946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c3212-41d3-487b-afee-bffe2fb521d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:36.632422 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c3212-41d3-487b-afee-bffe2fb521d3 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:36.642010 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1063/2111] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:38.117942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ae367-e3ea-46c7-85a9-1a70d8f7c449 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:38.119586 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-862ae367-e3ea-46c7-85a9-1a70d8f7c449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:38.228256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ae367-e3ea-46c7-85a9-1a70d8f7c449 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:38.234449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8187c4e-fa66-4dc5-b1e8-537c093b37d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:38.234844 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8187c4e-fa66-4dc5-b1e8-537c093b37d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:38.235225 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8187c4e-fa66-4dc5-b1e8-537c093b37d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:38.235598 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8187c4e-fa66-4dc5-b1e8-537c093b37d7 admin admin] PATH_INFO: `/v3/users/bf82d8704d004f25909fabc1b570146e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:38.236956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8187c4e-fa66-4dc5-b1e8-537c093b37d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf82d8704d004f25909fabc1b570146e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:38.239213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8187c4e-fa66-4dc5-b1e8-537c093b37d7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:38.298674 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d8187c4e-fa66-4dc5-b1e8-537c093b37d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf82d8704d004f25909fabc1b570146e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:38.307982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d8187c4e-fa66-4dc5-b1e8-537c093b37d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf82d8704d004f25909fabc1b570146e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:38.309087 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d8187c4e-fa66-4dc5-b1e8-537c093b37d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf82d8704d004f25909fabc1b570146e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:38.310176 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d8187c4e-fa66-4dc5-b1e8-537c093b37d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf82d8704d004f25909fabc1b570146e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:38.335708 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d8187c4e-fa66-4dc5-b1e8-537c093b37d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf82d8704d004f25909fabc1b570146e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:38.355811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1049/2112] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:38 2022] DELETE /identity/v3/users/bf82d8704d004f25909fabc1b570146e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:38.358697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c6535-82d7-4c70-93fb-f526191830c3 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:38.359455 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c6535-82d7-4c70-93fb-f526191830c3 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:38.359455 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c6535-82d7-4c70-93fb-f526191830c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:38.382166 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-c76c6535-82d7-4c70-93fb-f526191830c3 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:38.384193 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c76c6535-82d7-4c70-93fb-f526191830c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:38.453302 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1064/2113] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:38.460814 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8187c4e-fa66-4dc5-b1e8-537c093b37d7 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:38.463057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d8187c4e-fa66-4dc5-b1e8-537c093b37d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:38.528393 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8187c4e-fa66-4dc5-b1e8-537c093b37d7 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:38.532182 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21387f-e682-4a0d-83ff-d20193a173d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:38.532784 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21387f-e682-4a0d-83ff-d20193a173d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:38.533067 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21387f-e682-4a0d-83ff-d20193a173d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:38.533344 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21387f-e682-4a0d-83ff-d20193a173d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:38.539416 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:38.544211 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21387f-e682-4a0d-83ff-d20193a173d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:38.545727 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21387f-e682-4a0d-83ff-d20193a173d8 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:38.557035 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1050/2114] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:38.858411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c6535-82d7-4c70-93fb-f526191830c3 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:38.860154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c76c6535-82d7-4c70-93fb-f526191830c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:38.879444 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c6535-82d7-4c70-93fb-f526191830c3 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:38.885553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0298a-7dd9-44b1-aced-7caa07eea881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:38.886181 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0298a-7dd9-44b1-aced-7caa07eea881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:38.886458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0298a-7dd9-44b1-aced-7caa07eea881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:38.886728 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0298a-7dd9-44b1-aced-7caa07eea881 admin admin] PATH_INFO: `/v3/projects/497a30a0ad36400b9cfd082b6aa0abc5` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:38.887917 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0298a-7dd9-44b1-aced-7caa07eea881 admin admin] RBAC: Authorizing `identity:delete_project(project_id=497a30a0ad36400b9cfd082b6aa0abc5)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:38.890543 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0298a-7dd9-44b1-aced-7caa07eea881 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:38.928567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:38.940687 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8bf0298a-7dd9-44b1-aced-7caa07eea881 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '497a30a0ad36400b9cfd082b6aa0abc5', 'request_id': 'req-8bf0298a-7dd9-44b1-aced-7caa07eea881'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:38.948307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8bf0298a-7dd9-44b1-aced-7caa07eea881 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '497a30a0ad36400b9cfd082b6aa0abc5', 'request_id': 'req-8bf0298a-7dd9-44b1-aced-7caa07eea881'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:38.954465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8bf0298a-7dd9-44b1-aced-7caa07eea881 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '497a30a0ad36400b9cfd082b6aa0abc5', 'request_id': 'req-8bf0298a-7dd9-44b1-aced-7caa07eea881'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:38.968805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8bf0298a-7dd9-44b1-aced-7caa07eea881 admin admin] The token cache is being invalidate because project 497a30a0ad36400b9cfd082b6aa0abc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:38.969423 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8bf0298a-7dd9-44b1-aced-7caa07eea881 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:38.970193 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8bf0298a-7dd9-44b1-aced-7caa07eea881 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:38.972026 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1065/2115] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:38 2022] DELETE /identity/v3/projects/497a30a0ad36400b9cfd082b6aa0abc5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:38.977273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f239e-acda-464f-ace1-2a35bf3d5389 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:38.977558 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f239e-acda-464f-ace1-2a35bf3d5389 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:38.977895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f239e-acda-464f-ace1-2a35bf3d5389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:39.001311 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-0d5f239e-acda-464f-ace1-2a35bf3d5389 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:39.002584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0d5f239e-acda-464f-ace1-2a35bf3d5389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:39.070833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1051/2116] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:24:39.078777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50318d6e-121b-4fcb-beb8-ff0dab1dd805 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:39.079061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50318d6e-121b-4fcb-beb8-ff0dab1dd805 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:39.079798 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50318d6e-121b-4fcb-beb8-ff0dab1dd805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:39.100593 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-50318d6e-121b-4fcb-beb8-ff0dab1dd805 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:39.102112 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-50318d6e-121b-4fcb-beb8-ff0dab1dd805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:39.114313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1066/2117] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:24:39.119226 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f239e-acda-464f-ace1-2a35bf3d5389 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:39.136965 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f239e-acda-464f-ace1-2a35bf3d5389 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:39.144975 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd16dc2-cedb-48a5-8009-8e97d38a1f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:39.145652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd16dc2-cedb-48a5-8009-8e97d38a1f46 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:39.146433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd16dc2-cedb-48a5-8009-8e97d38a1f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:39.146433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd16dc2-cedb-48a5-8009-8e97d38a1f46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:39.147006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd16dc2-cedb-48a5-8009-8e97d38a1f46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:39.147308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd16dc2-cedb-48a5-8009-8e97d38a1f46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:39.149314 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd16dc2-cedb-48a5-8009-8e97d38a1f46 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:39.176802 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1052/2118] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:39.178453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50318d6e-121b-4fcb-beb8-ff0dab1dd805 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:39.194795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50318d6e-121b-4fcb-beb8-ff0dab1dd805 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:39.201099 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a85e78-7464-443b-b98f-ed9784164588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:39.203912 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a85e78-7464-443b-b98f-ed9784164588 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:39.204271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a85e78-7464-443b-b98f-ed9784164588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:39.204615 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a85e78-7464-443b-b98f-ed9784164588 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:39.205332 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a85e78-7464-443b-b98f-ed9784164588 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:39.207407 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a85e78-7464-443b-b98f-ed9784164588 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:39.255289 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1067/2119] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:39 2022] POST /identity/v3/projects => generated 385 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:39.261649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd16dc2-cedb-48a5-8009-8e97d38a1f46 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:39.286009 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd16dc2-cedb-48a5-8009-8e97d38a1f46 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:39.371665 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc66e69-1b78-420f-8a89-0588763f43d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:39.372455 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc66e69-1b78-420f-8a89-0588763f43d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:39.372741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc66e69-1b78-420f-8a89-0588763f43d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:39.373017 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc66e69-1b78-420f-8a89-0588763f43d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:39.373675 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc66e69-1b78-420f-8a89-0588763f43d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:39.376060 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc66e69-1b78-420f-8a89-0588763f43d8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:39.408311 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1053/2120] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:39 2022] POST /identity/v3/users => generated 293 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:39.412445 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a85e78-7464-443b-b98f-ed9784164588 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:39.425886 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a85e78-7464-443b-b98f-ed9784164588 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:39.431543 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cd0b7-37c2-4e56-888f-6550da484e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:39.432269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cd0b7-37c2-4e56-888f-6550da484e3b admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:39.432622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cd0b7-37c2-4e56-888f-6550da484e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:39.432905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cd0b7-37c2-4e56-888f-6550da484e3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:39.433553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0cd0b7-37c2-4e56-888f-6550da484e3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:39.435227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0cd0b7-37c2-4e56-888f-6550da484e3b admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:39.448756 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1068/2121] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:39.449874 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc66e69-1b78-420f-8a89-0588763f43d8 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:39.460339 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc66e69-1b78-420f-8a89-0588763f43d8 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:39.466710 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f526c-bbc7-4ff5-a69d-41de4fa5eb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:39.467428 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f526c-bbc7-4ff5-a69d-41de4fa5eb3d admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:39.467835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f526c-bbc7-4ff5-a69d-41de4fa5eb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:39.468206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f526c-bbc7-4ff5-a69d-41de4fa5eb3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:39.468958 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150f526c-bbc7-4ff5-a69d-41de4fa5eb3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:39.470576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150f526c-bbc7-4ff5-a69d-41de4fa5eb3d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:39.485623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1054/2122] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:39.487124 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cd0b7-37c2-4e56-888f-6550da484e3b admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:39.497804 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cd0b7-37c2-4e56-888f-6550da484e3b admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:39.502535 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f022071-0d79-412f-b44e-0bf04a12af84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:39.503203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f022071-0d79-412f-b44e-0bf04a12af84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:39.503476 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f022071-0d79-412f-b44e-0bf04a12af84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:39.503746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f022071-0d79-412f-b44e-0bf04a12af84 admin admin] PATH_INFO: `/v3/projects/de26cf9b68854106bcb8ee1ff9b6a132/users/7fd2d4181f5e45b88c8efc594d96a398/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:39.520305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f022071-0d79-412f-b44e-0bf04a12af84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de26cf9b68854106bcb8ee1ff9b6a132, user_id=7fd2d4181f5e45b88c8efc594d96a398, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:39.523017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f022071-0d79-412f-b44e-0bf04a12af84 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:39.537956 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1069/2123] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:39 2022] PUT /identity/v3/projects/de26cf9b68854106bcb8ee1ff9b6a132/users/7fd2d4181f5e45b88c8efc594d96a398/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:39.549155 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c2572-5341-4f46-8495-788730c2ce58 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:39.549640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c2572-5341-4f46-8495-788730c2ce58 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:39.550006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c2572-5341-4f46-8495-788730c2ce58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:39.580526 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e82c2572-5341-4f46-8495-788730c2ce58 None None] MFA Rules not processed for user `7fd2d4181f5e45b88c8efc594d96a398`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:39.581816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e82c2572-5341-4f46-8495-788730c2ce58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:39.634083 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:39.658516 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1055/2124] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:39.662824 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f022071-0d79-412f-b44e-0bf04a12af84 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:39.735033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f022071-0d79-412f-b44e-0bf04a12af84 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:39.740425 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9e0c8-76a6-4bdd-b098-3ba480bd105d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:39.741026 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9e0c8-76a6-4bdd-b098-3ba480bd105d admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:39.741304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9e0c8-76a6-4bdd-b098-3ba480bd105d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:39.741574 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9e0c8-76a6-4bdd-b098-3ba480bd105d admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:39.742200 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9e0c8-76a6-4bdd-b098-3ba480bd105d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:39.746983 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9e0c8-76a6-4bdd-b098-3ba480bd105d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:39.789029 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c2572-5341-4f46-8495-788730c2ce58 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:39.792209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e82c2572-5341-4f46-8495-788730c2ce58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:39.796147 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1070/2125] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:39 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:39.804070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9e0c8-76a6-4bdd-b098-3ba480bd105d admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:39.813673 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c2572-5341-4f46-8495-788730c2ce58 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:39.869814 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:39.894895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bd0d7-eb15-4448-8627-23f4764cc265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:39.895116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9e0c8-76a6-4bdd-b098-3ba480bd105d admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:39.895244 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bd0d7-eb15-4448-8627-23f4764cc265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:39.895244 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bd0d7-eb15-4448-8627-23f4764cc265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:39.895599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bd0d7-eb15-4448-8627-23f4764cc265 admin admin] PATH_INFO: `/v3/users/03bbd4546bba43e5a75b4ad669178237` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:39.896898 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9bd0d7-eb15-4448-8627-23f4764cc265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03bbd4546bba43e5a75b4ad669178237)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:39.898923 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9bd0d7-eb15-4448-8627-23f4764cc265 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:39.900894 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0b867-bd9b-4dc4-bce6-15697d5d55c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:39.901510 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0b867-bd9b-4dc4-bce6-15697d5d55c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:39.902144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0b867-bd9b-4dc4-bce6-15697d5d55c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:39.902424 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0b867-bd9b-4dc4-bce6-15697d5d55c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:39.903444 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0b867-bd9b-4dc4-bce6-15697d5d55c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:39.905438 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0b867-bd9b-4dc4-bce6-15697d5d55c4 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:39.932880 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1071/2126] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:39 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:24:39.937245 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0b867-bd9b-4dc4-bce6-15697d5d55c4 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:39.950866 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0b867-bd9b-4dc4-bce6-15697d5d55c4 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:39.955485 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5a9bd0d7-eb15-4448-8627-23f4764cc265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03bbd4546bba43e5a75b4ad669178237'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:39.957340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370ad40-2701-42f6-b6b9-6eeb32154664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:39.957676 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370ad40-2701-42f6-b6b9-6eeb32154664 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:39.957962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370ad40-2701-42f6-b6b9-6eeb32154664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:39.958822 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370ad40-2701-42f6-b6b9-6eeb32154664 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:39.959174 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b370ad40-2701-42f6-b6b9-6eeb32154664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:39.962935 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b370ad40-2701-42f6-b6b9-6eeb32154664 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:39.966702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5a9bd0d7-eb15-4448-8627-23f4764cc265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03bbd4546bba43e5a75b4ad669178237'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:39.967450 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5a9bd0d7-eb15-4448-8627-23f4764cc265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03bbd4546bba43e5a75b4ad669178237'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:39.968865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5a9bd0d7-eb15-4448-8627-23f4764cc265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03bbd4546bba43e5a75b4ad669178237'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:39.975561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1072/2127] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:39.979817 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370ad40-2701-42f6-b6b9-6eeb32154664 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:39.982254 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b370ad40-2701-42f6-b6b9-6eeb32154664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:39.984272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5a9bd0d7-eb15-4448-8627-23f4764cc265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03bbd4546bba43e5a75b4ad669178237'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:40.002751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370ad40-2701-42f6-b6b9-6eeb32154664 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:40.011123 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1056/2128] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:39 2022] DELETE /identity/v3/users/03bbd4546bba43e5a75b4ad669178237 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:40.012838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734466ce-b867-43d4-aa87-65a0d2e4083d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:40.013588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734466ce-b867-43d4-aa87-65a0d2e4083d admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:40.013974 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734466ce-b867-43d4-aa87-65a0d2e4083d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:40.014177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734466ce-b867-43d4-aa87-65a0d2e4083d admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:40.014875 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734466ce-b867-43d4-aa87-65a0d2e4083d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:40.016546 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734466ce-b867-43d4-aa87-65a0d2e4083d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:40.019005 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89056c4f-0ccf-462a-8541-7f42a286e731 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:40.019005 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89056c4f-0ccf-462a-8541-7f42a286e731 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:40.019005 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89056c4f-0ccf-462a-8541-7f42a286e731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:40.031679 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1073/2129] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:24:40.033053 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734466ce-b867-43d4-aa87-65a0d2e4083d admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:40.043393 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734466ce-b867-43d4-aa87-65a0d2e4083d admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:40.043734 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-89056c4f-0ccf-462a-8541-7f42a286e731 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:40.045146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-89056c4f-0ccf-462a-8541-7f42a286e731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:40.128553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc55a4-bfc6-4462-adae-bb1e3f187e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:40.129505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc55a4-bfc6-4462-adae-bb1e3f187e84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:40.129801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc55a4-bfc6-4462-adae-bb1e3f187e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:40.130076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc55a4-bfc6-4462-adae-bb1e3f187e84 admin admin] PATH_INFO: `/v3/projects/92cb1e1f43b242169ca1c6d906701e51/users/198983e7887640b38fd2a4c84875e4ce/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:40.146693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1057/2130] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:40.148236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfc55a4-bfc6-4462-adae-bb1e3f187e84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92cb1e1f43b242169ca1c6d906701e51, user_id=198983e7887640b38fd2a4c84875e4ce, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:40.151574 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfc55a4-bfc6-4462-adae-bb1e3f187e84 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:40.157954 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89056c4f-0ccf-462a-8541-7f42a286e731 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:40.159518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-89056c4f-0ccf-462a-8541-7f42a286e731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:40.168645 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1074/2131] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:40 2022] PUT /identity/v3/projects/92cb1e1f43b242169ca1c6d906701e51/users/198983e7887640b38fd2a4c84875e4ce/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:40.176524 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:40.181456 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe209520-0eb2-456b-8091-9fc88cb830d2 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:40.181782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe209520-0eb2-456b-8091-9fc88cb830d2 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:40.182064 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe209520-0eb2-456b-8091-9fc88cb830d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:40.223064 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-fe209520-0eb2-456b-8091-9fc88cb830d2 None None] MFA Rules not processed for user `198983e7887640b38fd2a4c84875e4ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:40.224329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-fe209520-0eb2-456b-8091-9fc88cb830d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:40.233853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89056c4f-0ccf-462a-8541-7f42a286e731 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:40.287379 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb439136-3c37-4c6e-8f18-ea0fedfe3b2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:40.288985 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb439136-3c37-4c6e-8f18-ea0fedfe3b2a service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:40.289382 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb439136-3c37-4c6e-8f18-ea0fedfe3b2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:40.289795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb439136-3c37-4c6e-8f18-ea0fedfe3b2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:40.293820 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1075/2132] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:24:40.298385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe209520-0eb2-456b-8091-9fc88cb830d2 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:40.331963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:40.392822 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb439136-3c37-4c6e-8f18-ea0fedfe3b2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:40.394484 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb439136-3c37-4c6e-8f18-ea0fedfe3b2a service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:40.402398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe209520-0eb2-456b-8091-9fc88cb830d2 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:40.408378 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1058/2133] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:40.411741 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a94841-81ca-4694-9af4-fc9cf3174874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:40.412445 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a94841-81ca-4694-9af4-fc9cf3174874 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:40.412723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a94841-81ca-4694-9af4-fc9cf3174874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:40.413303 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a94841-81ca-4694-9af4-fc9cf3174874 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:40.413476 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a94841-81ca-4694-9af4-fc9cf3174874 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:40.419667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a94841-81ca-4694-9af4-fc9cf3174874 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:40.492575 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb439136-3c37-4c6e-8f18-ea0fedfe3b2a service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:40.506212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1076/2134] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:40 2022] POST /identity/v3/projects => generated 383 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:40.506401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb439136-3c37-4c6e-8f18-ea0fedfe3b2a service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:40.609913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13266d69-ed78-4e33-b133-8240d3ceaf08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:40.610937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13266d69-ed78-4e33-b133-8240d3ceaf08 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:40.611344 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13266d69-ed78-4e33-b133-8240d3ceaf08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:40.612582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13266d69-ed78-4e33-b133-8240d3ceaf08 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:40.613446 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13266d69-ed78-4e33-b133-8240d3ceaf08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:40.617141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13266d69-ed78-4e33-b133-8240d3ceaf08 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:40.659982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1059/2135] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:40 2022] POST /identity/v3/users => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:40.661213 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a94841-81ca-4694-9af4-fc9cf3174874 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:40.674654 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a94841-81ca-4694-9af4-fc9cf3174874 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:40.689022 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b833ccf-155f-4cc9-903a-703b50924299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:40.689644 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b833ccf-155f-4cc9-903a-703b50924299 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:40.689921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b833ccf-155f-4cc9-903a-703b50924299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:40.690188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b833ccf-155f-4cc9-903a-703b50924299 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:40.690833 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b833ccf-155f-4cc9-903a-703b50924299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:40.692457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b833ccf-155f-4cc9-903a-703b50924299 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:40.711810 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1077/2136] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:40.715166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13266d69-ed78-4e33-b133-8240d3ceaf08 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:40.723884 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:40.730141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13266d69-ed78-4e33-b133-8240d3ceaf08 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:40.746018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d3f80-70b6-484e-899f-7da9b054f82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:40.746805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d3f80-70b6-484e-899f-7da9b054f82d admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:40.747181 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d3f80-70b6-484e-899f-7da9b054f82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:40.747680 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d3f80-70b6-484e-899f-7da9b054f82d admin admin] PATH_INFO: `/v3/projects/1218fa07fdaa4ca481f3f0b8f0847858/users/d62d54b643a34f56976845c04f35560d/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:40.785839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d3f80-70b6-484e-899f-7da9b054f82d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1218fa07fdaa4ca481f3f0b8f0847858, user_id=d62d54b643a34f56976845c04f35560d, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:40.794191 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d3f80-70b6-484e-899f-7da9b054f82d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:40.836376 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1060/2137] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:40 2022] PUT /identity/v3/projects/1218fa07fdaa4ca481f3f0b8f0847858/users/d62d54b643a34f56976845c04f35560d/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:40.839616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b833ccf-155f-4cc9-903a-703b50924299 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:40.851449 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b833ccf-155f-4cc9-903a-703b50924299 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:40.946854 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018a746-6b9a-44ac-9461-44bb9d0dc94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:40.947550 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018a746-6b9a-44ac-9461-44bb9d0dc94c admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:40.947934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018a746-6b9a-44ac-9461-44bb9d0dc94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:40.948275 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018a746-6b9a-44ac-9461-44bb9d0dc94c admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:40.949007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6018a746-6b9a-44ac-9461-44bb9d0dc94c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:40.950661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6018a746-6b9a-44ac-9461-44bb9d0dc94c admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:40.962052 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1078/2138] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:40.966289 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d3f80-70b6-484e-899f-7da9b054f82d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:40.976918 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d3f80-70b6-484e-899f-7da9b054f82d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:40.982239 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceca91e-907b-452d-af70-8afd0b78d168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:40.982908 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceca91e-907b-452d-af70-8afd0b78d168 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:40.983196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceca91e-907b-452d-af70-8afd0b78d168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:40.984822 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceca91e-907b-452d-af70-8afd0b78d168 admin admin] PATH_INFO: `/v3/projects/1218fa07fdaa4ca481f3f0b8f0847858/users/d62d54b643a34f56976845c04f35560d/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:40.986881 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceca91e-907b-452d-af70-8afd0b78d168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1218fa07fdaa4ca481f3f0b8f0847858, user_id=d62d54b643a34f56976845c04f35560d, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:40.989890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceca91e-907b-452d-af70-8afd0b78d168 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:41.006823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1061/2139] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:40 2022] PUT /identity/v3/projects/1218fa07fdaa4ca481f3f0b8f0847858/users/d62d54b643a34f56976845c04f35560d/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:41.017620 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81706a-3919-48d0-a730-0e7dac441cf7 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:41.017904 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81706a-3919-48d0-a730-0e7dac441cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:41.018176 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81706a-3919-48d0-a730-0e7dac441cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:41.048172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-ec81706a-3919-48d0-a730-0e7dac441cf7 None None] MFA Rules not processed for user `d62d54b643a34f56976845c04f35560d`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:41.053115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ec81706a-3919-48d0-a730-0e7dac441cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:41.089538 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceca91e-907b-452d-af70-8afd0b78d168 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:41.090401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-aceca91e-907b-452d-af70-8afd0b78d168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:41.143305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1079/2140] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:41.155570 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81706a-3919-48d0-a730-0e7dac441cf7 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:41.157395 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ec81706a-3919-48d0-a730-0e7dac441cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:41.181438 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:41.192580 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceca91e-907b-452d-af70-8afd0b78d168 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:41.201553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a114d6e-bb05-4b48-8bd2-0a7258f0dd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:41.202271 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a114d6e-bb05-4b48-8bd2-0a7258f0dd6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:41.202615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a114d6e-bb05-4b48-8bd2-0a7258f0dd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:41.202953 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a114d6e-bb05-4b48-8bd2-0a7258f0dd6c admin admin] PATH_INFO: `/v3/projects/f6f2757d16ea4e74b41d2c8ba52aa32a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:41.204222 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a114d6e-bb05-4b48-8bd2-0a7258f0dd6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6f2757d16ea4e74b41d2c8ba52aa32a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:41.206227 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a114d6e-bb05-4b48-8bd2-0a7258f0dd6c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:41.224242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81706a-3919-48d0-a730-0e7dac441cf7 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:41.228463 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15434f-9b05-4d94-b9b5-799f1df4950b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:41.229087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15434f-9b05-4d94-b9b5-799f1df4950b service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:41.229368 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15434f-9b05-4d94-b9b5-799f1df4950b service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:41.229642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15434f-9b05-4d94-b9b5-799f1df4950b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:41.231739 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ae15434f-9b05-4d94-b9b5-799f1df4950b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:41.258404 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0a114d6e-bb05-4b48-8bd2-0a7258f0dd6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6f2757d16ea4e74b41d2c8ba52aa32a', 'request_id': 'req-0a114d6e-bb05-4b48-8bd2-0a7258f0dd6c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:41.264787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0a114d6e-bb05-4b48-8bd2-0a7258f0dd6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6f2757d16ea4e74b41d2c8ba52aa32a', 'request_id': 'req-0a114d6e-bb05-4b48-8bd2-0a7258f0dd6c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:41.270374 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0a114d6e-bb05-4b48-8bd2-0a7258f0dd6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6f2757d16ea4e74b41d2c8ba52aa32a', 'request_id': 'req-0a114d6e-bb05-4b48-8bd2-0a7258f0dd6c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:41.283872 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0a114d6e-bb05-4b48-8bd2-0a7258f0dd6c admin admin] The token cache is being invalidate because project f6f2757d16ea4e74b41d2c8ba52aa32a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:41.284165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0a114d6e-bb05-4b48-8bd2-0a7258f0dd6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:41.284856 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0a114d6e-bb05-4b48-8bd2-0a7258f0dd6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:41.286385 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1062/2141] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:41 2022] DELETE /identity/v3/projects/f6f2757d16ea4e74b41d2c8ba52aa32a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:41.294051 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d279b80-a61e-4466-924a-bcadfc483395 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:41.294341 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d279b80-a61e-4466-924a-bcadfc483395 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:41.294527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d279b80-a61e-4466-924a-bcadfc483395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:41.296805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae15434f-9b05-4d94-b9b5-799f1df4950b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:41.298259 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae15434f-9b05-4d94-b9b5-799f1df4950b service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:41.308197 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1080/2142] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:24:41.313973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-2d279b80-a61e-4466-924a-bcadfc483395 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:41.315175 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2d279b80-a61e-4466-924a-bcadfc483395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:41.369044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:41.380445 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1063/2143] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:41.390449 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7b40f-9846-4d91-8740-7eceed84474c None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:41.390742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7b40f-9846-4d91-8740-7eceed84474c None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:41.391018 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7b40f-9846-4d91-8740-7eceed84474c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:41.415149 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-03e7b40f-9846-4d91-8740-7eceed84474c None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:41.417098 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-03e7b40f-9846-4d91-8740-7eceed84474c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:41.424988 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1081/2144] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:41.429332 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d279b80-a61e-4466-924a-bcadfc483395 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:41.439999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d279b80-a61e-4466-924a-bcadfc483395 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:41.444831 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218cb099-94b7-4565-a6d8-5a592a98dacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:41.445429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cb099-94b7-4565-a6d8-5a592a98dacb admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:41.445707 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cb099-94b7-4565-a6d8-5a592a98dacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:41.446516 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cb099-94b7-4565-a6d8-5a592a98dacb admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:41.447164 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218cb099-94b7-4565-a6d8-5a592a98dacb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:41.447468 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218cb099-94b7-4565-a6d8-5a592a98dacb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:41.449106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218cb099-94b7-4565-a6d8-5a592a98dacb admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:41.462737 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1064/2145] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:41.467628 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7b40f-9846-4d91-8740-7eceed84474c None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:41.478601 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7b40f-9846-4d91-8740-7eceed84474c None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:41.486704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fab775-97f2-4575-832c-7ecc2dca8307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:41.487457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fab775-97f2-4575-832c-7ecc2dca8307 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:41.487737 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fab775-97f2-4575-832c-7ecc2dca8307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:41.488046 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fab775-97f2-4575-832c-7ecc2dca8307 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:41.488689 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fab775-97f2-4575-832c-7ecc2dca8307 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:41.490596 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fab775-97f2-4575-832c-7ecc2dca8307 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:41.534810 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1082/2146] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:41 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:41.539132 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218cb099-94b7-4565-a6d8-5a592a98dacb admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:41.550123 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218cb099-94b7-4565-a6d8-5a592a98dacb admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:41.616591 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fbed0e-3254-4b92-a81e-aee65599b5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:41.617444 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fbed0e-3254-4b92-a81e-aee65599b5ee admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:41.618298 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fbed0e-3254-4b92-a81e-aee65599b5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:41.618584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fbed0e-3254-4b92-a81e-aee65599b5ee admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:41.619247 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fbed0e-3254-4b92-a81e-aee65599b5ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:41.621216 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fbed0e-3254-4b92-a81e-aee65599b5ee admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:41.643300 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1065/2147] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:41 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:41.647426 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fab775-97f2-4575-832c-7ecc2dca8307 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:41.661331 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fab775-97f2-4575-832c-7ecc2dca8307 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:41.666627 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe795d-4fc2-4756-992c-8caa531f337d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:41.667237 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe795d-4fc2-4756-992c-8caa531f337d admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:41.667520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe795d-4fc2-4756-992c-8caa531f337d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:41.667819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe795d-4fc2-4756-992c-8caa531f337d admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:41.668468 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe795d-4fc2-4756-992c-8caa531f337d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:41.670072 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe795d-4fc2-4756-992c-8caa531f337d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:41.681334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1083/2148] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:24:41.685633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fbed0e-3254-4b92-a81e-aee65599b5ee admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:41.695753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fbed0e-3254-4b92-a81e-aee65599b5ee admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:41.700514 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736907a-d060-46c1-be42-5f7ec72787f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:41.701110 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736907a-d060-46c1-be42-5f7ec72787f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:41.701391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736907a-d060-46c1-be42-5f7ec72787f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:41.701664 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736907a-d060-46c1-be42-5f7ec72787f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:41.702315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736907a-d060-46c1-be42-5f7ec72787f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:41.704097 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736907a-d060-46c1-be42-5f7ec72787f7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:41.715915 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1066/2149] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:41.719979 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe795d-4fc2-4756-992c-8caa531f337d admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:41.730550 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe795d-4fc2-4756-992c-8caa531f337d admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:41.735110 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7abe3b0-deb2-49cd-8b80-751f545c5eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:41.735780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abe3b0-deb2-49cd-8b80-751f545c5eac admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:41.736083 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abe3b0-deb2-49cd-8b80-751f545c5eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:41.736358 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abe3b0-deb2-49cd-8b80-751f545c5eac admin admin] PATH_INFO: `/v3/projects/951a4235bc114fbd8f247643652c4bdd/users/9ae7da8da21d4f219dc5ef32a48a6459/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:41.750739 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7abe3b0-deb2-49cd-8b80-751f545c5eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=951a4235bc114fbd8f247643652c4bdd, user_id=9ae7da8da21d4f219dc5ef32a48a6459, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:41.753412 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7abe3b0-deb2-49cd-8b80-751f545c5eac admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:41.765649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1084/2150] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:41 2022] PUT /identity/v3/projects/951a4235bc114fbd8f247643652c4bdd/users/9ae7da8da21d4f219dc5ef32a48a6459/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:41.776866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444e589-fd68-4d44-bf9c-ac32ce7bf49d None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:41.777155 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444e589-fd68-4d44-bf9c-ac32ce7bf49d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:41.777429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444e589-fd68-4d44-bf9c-ac32ce7bf49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:41.805991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6444e589-fd68-4d44-bf9c-ac32ce7bf49d None None] MFA Rules not processed for user `9ae7da8da21d4f219dc5ef32a48a6459`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:41.807186 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6444e589-fd68-4d44-bf9c-ac32ce7bf49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:41.871804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1067/2151] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:41.876427 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7abe3b0-deb2-49cd-8b80-751f545c5eac admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:41.938152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7abe3b0-deb2-49cd-8b80-751f545c5eac admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:41.942723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b2044-d95d-4669-863d-eed4ef17ac7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:41.943381 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b2044-d95d-4669-863d-eed4ef17ac7d admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:41.943659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b2044-d95d-4669-863d-eed4ef17ac7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:41.943956 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b2044-d95d-4669-863d-eed4ef17ac7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:41.944625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b2044-d95d-4669-863d-eed4ef17ac7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:41.946579 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b2044-d95d-4669-863d-eed4ef17ac7d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:41.962462 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:41.976559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1085/2152] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:41 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:41.981148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444e589-fd68-4d44-bf9c-ac32ce7bf49d None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:41.992086 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444e589-fd68-4d44-bf9c-ac32ce7bf49d None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:42.038603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:42.048970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa27b2-5a71-420d-8a7f-80ad7a03ffc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:42.049560 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa27b2-5a71-420d-8a7f-80ad7a03ffc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:42.049837 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa27b2-5a71-420d-8a7f-80ad7a03ffc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:42.050106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa27b2-5a71-420d-8a7f-80ad7a03ffc3 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:42.050737 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa27b2-5a71-420d-8a7f-80ad7a03ffc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:42.052674 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa27b2-5a71-420d-8a7f-80ad7a03ffc3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:42.074378 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1068/2153] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:41 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:42.078371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b2044-d95d-4669-863d-eed4ef17ac7d admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:42.089918 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b2044-d95d-4669-863d-eed4ef17ac7d admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:42.094592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99925eb2-59c2-48c0-af11-b516312d8f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:42.095191 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99925eb2-59c2-48c0-af11-b516312d8f15 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:42.095469 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99925eb2-59c2-48c0-af11-b516312d8f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:42.095739 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99925eb2-59c2-48c0-af11-b516312d8f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:42.096415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99925eb2-59c2-48c0-af11-b516312d8f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:42.098041 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99925eb2-59c2-48c0-af11-b516312d8f15 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:42.109657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1086/2154] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:42.113661 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa27b2-5a71-420d-8a7f-80ad7a03ffc3 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:42.123647 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa27b2-5a71-420d-8a7f-80ad7a03ffc3 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:42.128359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679730e8-69cd-4ad0-8276-ad6488209842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:42.128937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679730e8-69cd-4ad0-8276-ad6488209842 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:42.129214 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679730e8-69cd-4ad0-8276-ad6488209842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:42.129488 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679730e8-69cd-4ad0-8276-ad6488209842 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:42.130125 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679730e8-69cd-4ad0-8276-ad6488209842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:42.131722 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679730e8-69cd-4ad0-8276-ad6488209842 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:42.142395 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1069/2155] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:42.146609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99925eb2-59c2-48c0-af11-b516312d8f15 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:42.157688 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99925eb2-59c2-48c0-af11-b516312d8f15 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:42.162342 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cad214-9d01-43f5-bb27-4a71be93c2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:42.163106 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cad214-9d01-43f5-bb27-4a71be93c2c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:42.163442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cad214-9d01-43f5-bb27-4a71be93c2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:42.163798 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cad214-9d01-43f5-bb27-4a71be93c2c1 admin admin] PATH_INFO: `/v3/projects/91d67d37f7cc4d869a36393d925c79cb/users/704c51577aa7488bb5d276dd53a96c59/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:42.179216 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cad214-9d01-43f5-bb27-4a71be93c2c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91d67d37f7cc4d869a36393d925c79cb, user_id=704c51577aa7488bb5d276dd53a96c59, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:42.182160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cad214-9d01-43f5-bb27-4a71be93c2c1 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:42.195278 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1087/2156] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:42 2022] PUT /identity/v3/projects/91d67d37f7cc4d869a36393d925c79cb/users/704c51577aa7488bb5d276dd53a96c59/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:42.206165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c56dd-75fb-4615-b69c-f689d341b4bd None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:42.206717 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c56dd-75fb-4615-b69c-f689d341b4bd None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:42.206717 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c56dd-75fb-4615-b69c-f689d341b4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:42.237303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-058c56dd-75fb-4615-b69c-f689d341b4bd None None] MFA Rules not processed for user `704c51577aa7488bb5d276dd53a96c59`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:42.238533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-058c56dd-75fb-4615-b69c-f689d341b4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:42.306986 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1070/2157] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:42.311388 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cad214-9d01-43f5-bb27-4a71be93c2c1 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:42.378512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cad214-9d01-43f5-bb27-4a71be93c2c1 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:42.383109 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbc8f9-2373-4e9f-9a5b-c891b344f011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:42.384262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbc8f9-2373-4e9f-9a5b-c891b344f011 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:42.384541 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbc8f9-2373-4e9f-9a5b-c891b344f011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:42.384811 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbc8f9-2373-4e9f-9a5b-c891b344f011 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:42.385442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbc8f9-2373-4e9f-9a5b-c891b344f011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:42.387320 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbc8f9-2373-4e9f-9a5b-c891b344f011 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:42.414503 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1088/2158] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:42 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:24:42.418571 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c56dd-75fb-4615-b69c-f689d341b4bd None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:42.429151 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c56dd-75fb-4615-b69c-f689d341b4bd None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:42.485548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ea892-08f0-4655-ab17-e519fcc3d29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:42.486155 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ea892-08f0-4655-ab17-e519fcc3d29e admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:42.486438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ea892-08f0-4655-ab17-e519fcc3d29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:42.486715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ea892-08f0-4655-ab17-e519fcc3d29e admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:42.487359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ea892-08f0-4655-ab17-e519fcc3d29e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:42.489317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ea892-08f0-4655-ab17-e519fcc3d29e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:42.510674 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1071/2159] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:42 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:42.514657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbc8f9-2373-4e9f-9a5b-c891b344f011 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:42.525014 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbc8f9-2373-4e9f-9a5b-c891b344f011 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:42.529715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e26a61-558c-437b-acec-040778b0428f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:42.530307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e26a61-558c-437b-acec-040778b0428f admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:42.530586 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e26a61-558c-437b-acec-040778b0428f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:42.530859 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e26a61-558c-437b-acec-040778b0428f admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:42.531498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e26a61-558c-437b-acec-040778b0428f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:42.533142 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e26a61-558c-437b-acec-040778b0428f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:42.543609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1089/2160] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:42.547596 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ea892-08f0-4655-ab17-e519fcc3d29e admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:42.558220 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ea892-08f0-4655-ab17-e519fcc3d29e admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:42.562953 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a036f-35ce-467a-840c-06c59ee738a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:42.563602 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a036f-35ce-467a-840c-06c59ee738a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:42.563924 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a036f-35ce-467a-840c-06c59ee738a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:42.564208 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a036f-35ce-467a-840c-06c59ee738a0 admin admin] PATH_INFO: `/v3/projects/2b390c6aadae4ab5b266e08a1fc6b7c5/users/9f3a9c1061394e03aab05ecdf19b0dd2/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:42.579298 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a036f-35ce-467a-840c-06c59ee738a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b390c6aadae4ab5b266e08a1fc6b7c5, user_id=9f3a9c1061394e03aab05ecdf19b0dd2, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:42.582039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a036f-35ce-467a-840c-06c59ee738a0 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:42.594548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1072/2161] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:42 2022] PUT /identity/v3/projects/2b390c6aadae4ab5b266e08a1fc6b7c5/users/9f3a9c1061394e03aab05ecdf19b0dd2/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:42.598603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e26a61-558c-437b-acec-040778b0428f admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:42.608524 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e26a61-558c-437b-acec-040778b0428f admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:42.677523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b9c1f-a0bc-44b1-9a12-8bbeea5164e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:42.678139 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b9c1f-a0bc-44b1-9a12-8bbeea5164e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:42.678420 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b9c1f-a0bc-44b1-9a12-8bbeea5164e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:42.678706 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b9c1f-a0bc-44b1-9a12-8bbeea5164e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:42.679402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b9c1f-a0bc-44b1-9a12-8bbeea5164e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:42.681571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b9c1f-a0bc-44b1-9a12-8bbeea5164e7 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:42.691745 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1090/2162] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:42.696568 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a036f-35ce-467a-840c-06c59ee738a0 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:42.711036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a036f-35ce-467a-840c-06c59ee738a0 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:42.716120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7228e8-5b76-48f8-9c91-bb006e4fba8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:42.716764 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7228e8-5b76-48f8-9c91-bb006e4fba8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:42.717042 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7228e8-5b76-48f8-9c91-bb006e4fba8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:42.717313 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7228e8-5b76-48f8-9c91-bb006e4fba8f admin admin] PATH_INFO: `/v3/projects/2b390c6aadae4ab5b266e08a1fc6b7c5/users/9f3a9c1061394e03aab05ecdf19b0dd2/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:42.719254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7228e8-5b76-48f8-9c91-bb006e4fba8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b390c6aadae4ab5b266e08a1fc6b7c5, user_id=9f3a9c1061394e03aab05ecdf19b0dd2, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:42.721929 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7228e8-5b76-48f8-9c91-bb006e4fba8f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:42.732023 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1073/2163] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:42 2022] PUT /identity/v3/projects/2b390c6aadae4ab5b266e08a1fc6b7c5/users/9f3a9c1061394e03aab05ecdf19b0dd2/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:42.799796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d4fd0-15ad-4c94-be7c-cfeb2b81fa81 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:42.800101 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d4fd0-15ad-4c94-be7c-cfeb2b81fa81 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:42.800384 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d4fd0-15ad-4c94-be7c-cfeb2b81fa81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:42.828504 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-da9d4fd0-15ad-4c94-be7c-cfeb2b81fa81 None None] MFA Rules not processed for user `9f3a9c1061394e03aab05ecdf19b0dd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:42.829720 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-da9d4fd0-15ad-4c94-be7c-cfeb2b81fa81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:42.848574 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:42.898812 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1091/2164] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:42.909934 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7228e8-5b76-48f8-9c91-bb006e4fba8f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:42.911712 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ab7228e8-5b76-48f8-9c91-bb006e4fba8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:42.928497 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:42.973478 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7228e8-5b76-48f8-9c91-bb006e4fba8f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:42.977761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785a513-b868-46de-b2b4-242e643a03a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:42.978317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785a513-b868-46de-b2b4-242e643a03a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:42.978598 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785a513-b868-46de-b2b4-242e643a03a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:42.978868 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785a513-b868-46de-b2b4-242e643a03a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:43.036059 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785a513-b868-46de-b2b4-242e643a03a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:43.037533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785a513-b868-46de-b2b4-242e643a03a3 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:43.046846 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1074/2165] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:44.971970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d4fd0-15ad-4c94-be7c-cfeb2b81fa81 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:44.973484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-da9d4fd0-15ad-4c94-be7c-cfeb2b81fa81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:44.988579 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d4fd0-15ad-4c94-be7c-cfeb2b81fa81 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:44.993390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53928ebf-3d30-40cd-a5d7-c22cc59d0d18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:44.994054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53928ebf-3d30-40cd-a5d7-c22cc59d0d18 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:44.994313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53928ebf-3d30-40cd-a5d7-c22cc59d0d18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:44.994586 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53928ebf-3d30-40cd-a5d7-c22cc59d0d18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:44.996582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-53928ebf-3d30-40cd-a5d7-c22cc59d0d18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:45.058296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53928ebf-3d30-40cd-a5d7-c22cc59d0d18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:45.059727 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53928ebf-3d30-40cd-a5d7-c22cc59d0d18 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:45.069598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1092/2166] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:24:45.616117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785a513-b868-46de-b2b4-242e643a03a3 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:45.626219 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785a513-b868-46de-b2b4-242e643a03a3 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:45.632906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0da23-2dca-4291-b593-d4259e302ed1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:45.633759 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0da23-2dca-4291-b593-d4259e302ed1 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:45.634036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0da23-2dca-4291-b593-d4259e302ed1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:45.634311 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0da23-2dca-4291-b593-d4259e302ed1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:45.720140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0da23-2dca-4291-b593-d4259e302ed1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:45.721566 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0da23-2dca-4291-b593-d4259e302ed1 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:45.732387 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1075/2167] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:45.845969 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53928ebf-3d30-40cd-a5d7-c22cc59d0d18 service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:45.855408 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53928ebf-3d30-40cd-a5d7-c22cc59d0d18 service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:45.859808 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7dadbc-82c0-4eb7-964f-00a9f529d2f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:45.860589 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7dadbc-82c0-4eb7-964f-00a9f529d2f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:45.861002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7dadbc-82c0-4eb7-964f-00a9f529d2f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:45.861397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7dadbc-82c0-4eb7-964f-00a9f529d2f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:45.864207 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-bf7dadbc-82c0-4eb7-964f-00a9f529d2f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:45.940163 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7dadbc-82c0-4eb7-964f-00a9f529d2f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:45.941770 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7dadbc-82c0-4eb7-964f-00a9f529d2f7 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:45.956766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1093/2168] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:45 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:24:46.381621 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0da23-2dca-4291-b593-d4259e302ed1 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:46.390967 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0da23-2dca-4291-b593-d4259e302ed1 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:46.395268 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46881df-d692-4422-b678-e3a37e9a4dd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:46.395933 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46881df-d692-4422-b678-e3a37e9a4dd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:46.396242 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46881df-d692-4422-b678-e3a37e9a4dd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:46.396514 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46881df-d692-4422-b678-e3a37e9a4dd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:46.410510 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46881df-d692-4422-b678-e3a37e9a4dd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:46.412048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46881df-d692-4422-b678-e3a37e9a4dd7 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:46.423789 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1076/2169] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:48.981742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7dadbc-82c0-4eb7-964f-00a9f529d2f7 service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:48.983492 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-bf7dadbc-82c0-4eb7-964f-00a9f529d2f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:49.076180 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:49.098934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7dadbc-82c0-4eb7-964f-00a9f529d2f7 service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:49.104182 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617de8d-bf13-4a57-aaa6-071a41e9555f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:49.104872 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617de8d-bf13-4a57-aaa6-071a41e9555f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:49.105233 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617de8d-bf13-4a57-aaa6-071a41e9555f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:49.105514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617de8d-bf13-4a57-aaa6-071a41e9555f admin admin] PATH_INFO: `/v3/users/aa5239f9f3024874847b609ea8f98575` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:49.106697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617de8d-bf13-4a57-aaa6-071a41e9555f admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa5239f9f3024874847b609ea8f98575)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:49.108591 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617de8d-bf13-4a57-aaa6-071a41e9555f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:49.194884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-7617de8d-bf13-4a57-aaa6-071a41e9555f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa5239f9f3024874847b609ea8f98575'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:49.196119 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-7617de8d-bf13-4a57-aaa6-071a41e9555f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa5239f9f3024874847b609ea8f98575'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:49.197976 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-7617de8d-bf13-4a57-aaa6-071a41e9555f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa5239f9f3024874847b609ea8f98575'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:49.210031 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-7617de8d-bf13-4a57-aaa6-071a41e9555f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa5239f9f3024874847b609ea8f98575'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:49.229894 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-7617de8d-bf13-4a57-aaa6-071a41e9555f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa5239f9f3024874847b609ea8f98575'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:49.253146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1094/2170] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:48 2022] DELETE /identity/v3/users/aa5239f9f3024874847b609ea8f98575 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:49.260827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716841ac-3e3c-4984-8cf3-da8f81daaa5c None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:49.262138 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716841ac-3e3c-4984-8cf3-da8f81daaa5c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:49.262640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716841ac-3e3c-4984-8cf3-da8f81daaa5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:49.313861 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-716841ac-3e3c-4984-8cf3-da8f81daaa5c None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:49.316373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-716841ac-3e3c-4984-8cf3-da8f81daaa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:49.382673 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:49.410614 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1077/2171] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:49.420461 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617de8d-bf13-4a57-aaa6-071a41e9555f admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:49.421909 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-7617de8d-bf13-4a57-aaa6-071a41e9555f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:49.506571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617de8d-bf13-4a57-aaa6-071a41e9555f admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:49.521322 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94759905-caf3-4f1c-9cd5-6d550f19e909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:49.521937 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94759905-caf3-4f1c-9cd5-6d550f19e909 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:49.522854 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94759905-caf3-4f1c-9cd5-6d550f19e909 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:49.523135 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94759905-caf3-4f1c-9cd5-6d550f19e909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:49.534349 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94759905-caf3-4f1c-9cd5-6d550f19e909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:49.535898 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94759905-caf3-4f1c-9cd5-6d550f19e909 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:49.549271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1095/2172] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:49.948546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716841ac-3e3c-4984-8cf3-da8f81daaa5c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:49.950430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-716841ac-3e3c-4984-8cf3-da8f81daaa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.000824 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716841ac-3e3c-4984-8cf3-da8f81daaa5c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:50.005825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc916b-f682-40e6-b2ed-a8a5f4001dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:50.006399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc916b-f682-40e6-b2ed-a8a5f4001dfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.006682 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc916b-f682-40e6-b2ed-a8a5f4001dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.006956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc916b-f682-40e6-b2ed-a8a5f4001dfa admin admin] PATH_INFO: `/v3/projects/6c14c7e4417f4da0b4edc45ca7bcb176` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.008163 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc916b-f682-40e6-b2ed-a8a5f4001dfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c14c7e4417f4da0b4edc45ca7bcb176)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:50.010169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc916b-f682-40e6-b2ed-a8a5f4001dfa admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:50.047309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94759905-caf3-4f1c-9cd5-6d550f19e909 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:50.048910 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-94759905-caf3-4f1c-9cd5-6d550f19e909 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.063205 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94759905-caf3-4f1c-9cd5-6d550f19e909 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:50.073204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21966e23-8609-473a-935f-0e34792f88d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:50.074145 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21966e23-8609-473a-935f-0e34792f88d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.074595 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21966e23-8609-473a-935f-0e34792f88d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.075226 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21966e23-8609-473a-935f-0e34792f88d1 admin admin] PATH_INFO: `/v3/users/7fd2d4181f5e45b88c8efc594d96a398` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.077209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21966e23-8609-473a-935f-0e34792f88d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fd2d4181f5e45b88c8efc594d96a398)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:50.079448 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21966e23-8609-473a-935f-0e34792f88d1 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:50.086388 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7efc916b-f682-40e6-b2ed-a8a5f4001dfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c14c7e4417f4da0b4edc45ca7bcb176', 'request_id': 'req-7efc916b-f682-40e6-b2ed-a8a5f4001dfa'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:50.101002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7efc916b-f682-40e6-b2ed-a8a5f4001dfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c14c7e4417f4da0b4edc45ca7bcb176', 'request_id': 'req-7efc916b-f682-40e6-b2ed-a8a5f4001dfa'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:50.110423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7efc916b-f682-40e6-b2ed-a8a5f4001dfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c14c7e4417f4da0b4edc45ca7bcb176', 'request_id': 'req-7efc916b-f682-40e6-b2ed-a8a5f4001dfa'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:50.129055 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7efc916b-f682-40e6-b2ed-a8a5f4001dfa admin admin] The token cache is being invalidate because project 6c14c7e4417f4da0b4edc45ca7bcb176 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:50.129407 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7efc916b-f682-40e6-b2ed-a8a5f4001dfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:50.130084 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7efc916b-f682-40e6-b2ed-a8a5f4001dfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:50.131971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1078/2173] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:49 2022] DELETE /identity/v3/projects/6c14c7e4417f4da0b4edc45ca7bcb176 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:50.172725 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-21966e23-8609-473a-935f-0e34792f88d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fd2d4181f5e45b88c8efc594d96a398'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:50.174174 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-21966e23-8609-473a-935f-0e34792f88d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fd2d4181f5e45b88c8efc594d96a398'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:50.174897 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-21966e23-8609-473a-935f-0e34792f88d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fd2d4181f5e45b88c8efc594d96a398'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:50.185176 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-21966e23-8609-473a-935f-0e34792f88d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fd2d4181f5e45b88c8efc594d96a398'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:50.194339 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:50.206699 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-21966e23-8609-473a-935f-0e34792f88d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fd2d4181f5e45b88c8efc594d96a398'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:50.230805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1096/2174] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:50 2022] DELETE /identity/v3/users/7fd2d4181f5e45b88c8efc594d96a398 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:50.238853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5d3d2-be47-4007-a200-e4d6d81a4fe1 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.239342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5d3d2-be47-4007-a200-e4d6d81a4fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.239735 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5d3d2-be47-4007-a200-e4d6d81a4fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.312775 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-87a5d3d2-be47-4007-a200-e4d6d81a4fe1 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:50.321831 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-87a5d3d2-be47-4007-a200-e4d6d81a4fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.401481 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1079/2175] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:50.412439 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21966e23-8609-473a-935f-0e34792f88d1 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:50.414265 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-21966e23-8609-473a-935f-0e34792f88d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.479510 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21966e23-8609-473a-935f-0e34792f88d1 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:50.483284 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e09ac4-51ce-48fb-a140-39e040c6ce2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:50.484059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e09ac4-51ce-48fb-a140-39e040c6ce2c service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.484481 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e09ac4-51ce-48fb-a140-39e040c6ce2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.484852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e09ac4-51ce-48fb-a140-39e040c6ce2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.496867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e09ac4-51ce-48fb-a140-39e040c6ce2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:50.498411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e09ac4-51ce-48fb-a140-39e040c6ce2c service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:50.510023 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1097/2176] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:50.548106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5d3d2-be47-4007-a200-e4d6d81a4fe1 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:50.551496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-87a5d3d2-be47-4007-a200-e4d6d81a4fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.574166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5d3d2-be47-4007-a200-e4d6d81a4fe1 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:50.581429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94546a2-54a2-45d2-92ea-66b16a7451a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:50.582240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94546a2-54a2-45d2-92ea-66b16a7451a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.582619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94546a2-54a2-45d2-92ea-66b16a7451a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.582993 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94546a2-54a2-45d2-92ea-66b16a7451a0 admin admin] PATH_INFO: `/v3/users/9ae7da8da21d4f219dc5ef32a48a6459` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.584696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94546a2-54a2-45d2-92ea-66b16a7451a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ae7da8da21d4f219dc5ef32a48a6459)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:50.589069 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94546a2-54a2-45d2-92ea-66b16a7451a0 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:50.645805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d94546a2-54a2-45d2-92ea-66b16a7451a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ae7da8da21d4f219dc5ef32a48a6459'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:50.665000 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d94546a2-54a2-45d2-92ea-66b16a7451a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ae7da8da21d4f219dc5ef32a48a6459'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:50.665654 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d94546a2-54a2-45d2-92ea-66b16a7451a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ae7da8da21d4f219dc5ef32a48a6459'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:50.676194 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d94546a2-54a2-45d2-92ea-66b16a7451a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ae7da8da21d4f219dc5ef32a48a6459'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:50.695732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d94546a2-54a2-45d2-92ea-66b16a7451a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ae7da8da21d4f219dc5ef32a48a6459'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:50.698510 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:50.720714 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1080/2177] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:50 2022] DELETE /identity/v3/users/9ae7da8da21d4f219dc5ef32a48a6459 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:50.726923 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c9b64-80b2-41a8-ac8c-5f43f581c930 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.727265 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c9b64-80b2-41a8-ac8c-5f43f581c930 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.727663 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c9b64-80b2-41a8-ac8c-5f43f581c930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.756096 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-856c9b64-80b2-41a8-ac8c-5f43f581c930 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:50.757625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-856c9b64-80b2-41a8-ac8c-5f43f581c930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.864985 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1098/2178] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:50.881159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94546a2-54a2-45d2-92ea-66b16a7451a0 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:50.882296 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d94546a2-54a2-45d2-92ea-66b16a7451a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.913449 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c9b64-80b2-41a8-ac8c-5f43f581c930 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:50.920199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-856c9b64-80b2-41a8-ac8c-5f43f581c930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:50.960013 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c9b64-80b2-41a8-ac8c-5f43f581c930 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:50.977406 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e5790-68dd-4074-bd19-c014dff23f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:50.978056 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e5790-68dd-4074-bd19-c014dff23f1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:50.978374 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e5790-68dd-4074-bd19-c014dff23f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:50.984281 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e5790-68dd-4074-bd19-c014dff23f1a admin admin] PATH_INFO: `/v3/projects/de26cf9b68854106bcb8ee1ff9b6a132` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:50.985663 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e5790-68dd-4074-bd19-c014dff23f1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=de26cf9b68854106bcb8ee1ff9b6a132)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:50.987712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e5790-68dd-4074-bd19-c014dff23f1a admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:51.015265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94546a2-54a2-45d2-92ea-66b16a7451a0 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:51.022667 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429ccd9-aa26-416a-bb29-fdb4bc07103c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:51.023949 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429ccd9-aa26-416a-bb29-fdb4bc07103c service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:51.024431 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429ccd9-aa26-416a-bb29-fdb4bc07103c service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:51.025062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429ccd9-aa26-416a-bb29-fdb4bc07103c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:51.039339 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429ccd9-aa26-416a-bb29-fdb4bc07103c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:51.041869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429ccd9-aa26-416a-bb29-fdb4bc07103c service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:51.090354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1081/2179] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:51.100555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-da9e5790-68dd-4074-bd19-c014dff23f1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de26cf9b68854106bcb8ee1ff9b6a132', 'request_id': 'req-da9e5790-68dd-4074-bd19-c014dff23f1a'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:51.108902 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-da9e5790-68dd-4074-bd19-c014dff23f1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de26cf9b68854106bcb8ee1ff9b6a132', 'request_id': 'req-da9e5790-68dd-4074-bd19-c014dff23f1a'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:51.122592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-da9e5790-68dd-4074-bd19-c014dff23f1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de26cf9b68854106bcb8ee1ff9b6a132', 'request_id': 'req-da9e5790-68dd-4074-bd19-c014dff23f1a'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:51.160744 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-da9e5790-68dd-4074-bd19-c014dff23f1a admin admin] The token cache is being invalidate because project de26cf9b68854106bcb8ee1ff9b6a132 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:51.161270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-da9e5790-68dd-4074-bd19-c014dff23f1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:51.163680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-da9e5790-68dd-4074-bd19-c014dff23f1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:51.167648 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1099/2180] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:50 2022] DELETE /identity/v3/projects/de26cf9b68854106bcb8ee1ff9b6a132 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:51.171817 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429ccd9-aa26-416a-bb29-fdb4bc07103c service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.189237 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429ccd9-aa26-416a-bb29-fdb4bc07103c service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:51.366276 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135784f-9c5d-4112-8bac-6ac84ae29a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:51.366925 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135784f-9c5d-4112-8bac-6ac84ae29a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:51.367217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135784f-9c5d-4112-8bac-6ac84ae29a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:51.367448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135784f-9c5d-4112-8bac-6ac84ae29a2d admin admin] PATH_INFO: `/v3/users/198983e7887640b38fd2a4c84875e4ce` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:51.369263 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135784f-9c5d-4112-8bac-6ac84ae29a2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=198983e7887640b38fd2a4c84875e4ce)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:51.371803 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135784f-9c5d-4112-8bac-6ac84ae29a2d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:51.479094 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9135784f-9c5d-4112-8bac-6ac84ae29a2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '198983e7887640b38fd2a4c84875e4ce'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:51.493574 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9135784f-9c5d-4112-8bac-6ac84ae29a2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '198983e7887640b38fd2a4c84875e4ce'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:51.494673 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9135784f-9c5d-4112-8bac-6ac84ae29a2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '198983e7887640b38fd2a4c84875e4ce'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:51.497980 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9135784f-9c5d-4112-8bac-6ac84ae29a2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '198983e7887640b38fd2a4c84875e4ce'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:51.520143 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9135784f-9c5d-4112-8bac-6ac84ae29a2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '198983e7887640b38fd2a4c84875e4ce'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:51.537669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1082/2181] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:51 2022] DELETE /identity/v3/users/198983e7887640b38fd2a4c84875e4ce => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:51.632113 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e5790-68dd-4074-bd19-c014dff23f1a admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.633885 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-da9e5790-68dd-4074-bd19-c014dff23f1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:51.649370 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e5790-68dd-4074-bd19-c014dff23f1a admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:51.659520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135784f-9c5d-4112-8bac-6ac84ae29a2d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.661060 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9135784f-9c5d-4112-8bac-6ac84ae29a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:51.664810 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:51.691343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:51.751578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df43c57-308f-4567-9fa1-db7ff5b360f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:51.752149 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df43c57-308f-4567-9fa1-db7ff5b360f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:51.752463 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df43c57-308f-4567-9fa1-db7ff5b360f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:51.752745 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df43c57-308f-4567-9fa1-db7ff5b360f3 admin admin] PATH_INFO: `/v3/users/aada82b4250745ab956466f29d1ddfb1` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:51.753905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df43c57-308f-4567-9fa1-db7ff5b360f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aada82b4250745ab956466f29d1ddfb1)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:51.755400 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135784f-9c5d-4112-8bac-6ac84ae29a2d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:51.756781 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df43c57-308f-4567-9fa1-db7ff5b360f3 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:51.762328 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14701a84-cba6-4835-8bcb-388a921c75a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:51.763083 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14701a84-cba6-4835-8bcb-388a921c75a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:51.763238 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14701a84-cba6-4835-8bcb-388a921c75a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:51.763513 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14701a84-cba6-4835-8bcb-388a921c75a6 admin admin] PATH_INFO: `/v3/projects/951a4235bc114fbd8f247643652c4bdd` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:51.764707 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14701a84-cba6-4835-8bcb-388a921c75a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=951a4235bc114fbd8f247643652c4bdd)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:51.766627 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14701a84-cba6-4835-8bcb-388a921c75a6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:51.822502 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4df43c57-308f-4567-9fa1-db7ff5b360f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aada82b4250745ab956466f29d1ddfb1'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:51.823575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4df43c57-308f-4567-9fa1-db7ff5b360f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aada82b4250745ab956466f29d1ddfb1'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:51.824207 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4df43c57-308f-4567-9fa1-db7ff5b360f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aada82b4250745ab956466f29d1ddfb1'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:51.824585 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-14701a84-cba6-4835-8bcb-388a921c75a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '951a4235bc114fbd8f247643652c4bdd', 'request_id': 'req-14701a84-cba6-4835-8bcb-388a921c75a6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:51.832776 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4df43c57-308f-4567-9fa1-db7ff5b360f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aada82b4250745ab956466f29d1ddfb1'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:51.832966 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-14701a84-cba6-4835-8bcb-388a921c75a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '951a4235bc114fbd8f247643652c4bdd', 'request_id': 'req-14701a84-cba6-4835-8bcb-388a921c75a6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:51.839849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-14701a84-cba6-4835-8bcb-388a921c75a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '951a4235bc114fbd8f247643652c4bdd', 'request_id': 'req-14701a84-cba6-4835-8bcb-388a921c75a6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:51.848251 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4df43c57-308f-4567-9fa1-db7ff5b360f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aada82b4250745ab956466f29d1ddfb1'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:51.855634 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-14701a84-cba6-4835-8bcb-388a921c75a6 admin admin] The token cache is being invalidate because project 951a4235bc114fbd8f247643652c4bdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:51.856054 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-14701a84-cba6-4835-8bcb-388a921c75a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:51.857111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-14701a84-cba6-4835-8bcb-388a921c75a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:51.859083 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1083/2182] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:51 2022] DELETE /identity/v3/projects/951a4235bc114fbd8f247643652c4bdd => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:51.863190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14701a84-cba6-4835-8bcb-388a921c75a6 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.863576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1100/2183] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:51 2022] DELETE /identity/v3/users/aada82b4250745ab956466f29d1ddfb1 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:51.864831 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-14701a84-cba6-4835-8bcb-388a921c75a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:51.918521 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df43c57-308f-4567-9fa1-db7ff5b360f3 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:51.919993 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-4df43c57-308f-4567-9fa1-db7ff5b360f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:51.958359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14701a84-cba6-4835-8bcb-388a921c75a6 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.006805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df43c57-308f-4567-9fa1-db7ff5b360f3 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.011530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c6ff48-bd90-4d3f-953c-f3ce64b54d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.012193 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6ff48-bd90-4d3f-953c-f3ce64b54d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.012853 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6ff48-bd90-4d3f-953c-f3ce64b54d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.012965 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:52.013062 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6ff48-bd90-4d3f-953c-f3ce64b54d50 admin admin] PATH_INFO: `/v3/projects/92cb1e1f43b242169ca1c6d906701e51` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.014378 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c6ff48-bd90-4d3f-953c-f3ce64b54d50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92cb1e1f43b242169ca1c6d906701e51)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.016588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c6ff48-bd90-4d3f-953c-f3ce64b54d50 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.031166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192f9d6-58b7-4089-95e0-2f82c0c5d2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.031858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192f9d6-58b7-4089-95e0-2f82c0c5d2ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.032222 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192f9d6-58b7-4089-95e0-2f82c0c5d2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.032505 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192f9d6-58b7-4089-95e0-2f82c0c5d2ea admin admin] PATH_INFO: `/v3/users/704c51577aa7488bb5d276dd53a96c59` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.033659 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192f9d6-58b7-4089-95e0-2f82c0c5d2ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=704c51577aa7488bb5d276dd53a96c59)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.035672 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192f9d6-58b7-4089-95e0-2f82c0c5d2ea admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.062441 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:52.080361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e9c6ff48-bd90-4d3f-953c-f3ce64b54d50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92cb1e1f43b242169ca1c6d906701e51', 'request_id': 'req-e9c6ff48-bd90-4d3f-953c-f3ce64b54d50'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.088638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e9c6ff48-bd90-4d3f-953c-f3ce64b54d50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92cb1e1f43b242169ca1c6d906701e51', 'request_id': 'req-e9c6ff48-bd90-4d3f-953c-f3ce64b54d50'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.095124 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e9c6ff48-bd90-4d3f-953c-f3ce64b54d50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92cb1e1f43b242169ca1c6d906701e51', 'request_id': 'req-e9c6ff48-bd90-4d3f-953c-f3ce64b54d50'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.105415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c192f9d6-58b7-4089-95e0-2f82c0c5d2ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '704c51577aa7488bb5d276dd53a96c59'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.112262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e9c6ff48-bd90-4d3f-953c-f3ce64b54d50 admin admin] The token cache is being invalidate because project 92cb1e1f43b242169ca1c6d906701e51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:52.112708 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e9c6ff48-bd90-4d3f-953c-f3ce64b54d50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.113245 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e9c6ff48-bd90-4d3f-953c-f3ce64b54d50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.114549 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c192f9d6-58b7-4089-95e0-2f82c0c5d2ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '704c51577aa7488bb5d276dd53a96c59'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.114968 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1101/2184] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:51 2022] DELETE /identity/v3/projects/92cb1e1f43b242169ca1c6d906701e51 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:52.115677 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c6ff48-bd90-4d3f-953c-f3ce64b54d50 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.117011 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e9c6ff48-bd90-4d3f-953c-f3ce64b54d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:52.118143 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c192f9d6-58b7-4089-95e0-2f82c0c5d2ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '704c51577aa7488bb5d276dd53a96c59'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.119327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c192f9d6-58b7-4089-95e0-2f82c0c5d2ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '704c51577aa7488bb5d276dd53a96c59'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.132696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c192f9d6-58b7-4089-95e0-2f82c0c5d2ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '704c51577aa7488bb5d276dd53a96c59'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.146052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1084/2185] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:51 2022] DELETE /identity/v3/users/704c51577aa7488bb5d276dd53a96c59 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:52.146624 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192f9d6-58b7-4089-95e0-2f82c0c5d2ea admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.147975 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c192f9d6-58b7-4089-95e0-2f82c0c5d2ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:52.190150 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c6ff48-bd90-4d3f-953c-f3ce64b54d50 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.225014 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192f9d6-58b7-4089-95e0-2f82c0c5d2ea admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.230954 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0413515-861f-4fa1-b987-d58d7c34cb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.231718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0413515-861f-4fa1-b987-d58d7c34cb01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.232177 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0413515-861f-4fa1-b987-d58d7c34cb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.232686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0413515-861f-4fa1-b987-d58d7c34cb01 admin admin] PATH_INFO: `/v3/users/d62d54b643a34f56976845c04f35560d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.233993 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0413515-861f-4fa1-b987-d58d7c34cb01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d62d54b643a34f56976845c04f35560d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.236302 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0413515-861f-4fa1-b987-d58d7c34cb01 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.255507 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e88f9-d0e8-47cb-b004-adeab49999eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.256427 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e88f9-d0e8-47cb-b004-adeab49999eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.256829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e88f9-d0e8-47cb-b004-adeab49999eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.257247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e88f9-d0e8-47cb-b004-adeab49999eb admin admin] PATH_INFO: `/v3/projects/8fd848b22f854563a0ab63ec0971b595` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.258484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e88f9-d0e8-47cb-b004-adeab49999eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fd848b22f854563a0ab63ec0971b595)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.261227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e88f9-d0e8-47cb-b004-adeab49999eb admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.291505 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a0413515-861f-4fa1-b987-d58d7c34cb01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd62d54b643a34f56976845c04f35560d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.301853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a0413515-861f-4fa1-b987-d58d7c34cb01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd62d54b643a34f56976845c04f35560d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.302475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a0413515-861f-4fa1-b987-d58d7c34cb01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd62d54b643a34f56976845c04f35560d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.303090 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a0413515-861f-4fa1-b987-d58d7c34cb01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd62d54b643a34f56976845c04f35560d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.318574 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:52.320818 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a0413515-861f-4fa1-b987-d58d7c34cb01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd62d54b643a34f56976845c04f35560d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.322805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a60e88f9-d0e8-47cb-b004-adeab49999eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fd848b22f854563a0ab63ec0971b595', 'request_id': 'req-a60e88f9-d0e8-47cb-b004-adeab49999eb'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.330837 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a60e88f9-d0e8-47cb-b004-adeab49999eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fd848b22f854563a0ab63ec0971b595', 'request_id': 'req-a60e88f9-d0e8-47cb-b004-adeab49999eb'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.338366 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1085/2186] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:52 2022] DELETE /identity/v3/users/d62d54b643a34f56976845c04f35560d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:52.338544 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a60e88f9-d0e8-47cb-b004-adeab49999eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fd848b22f854563a0ab63ec0971b595', 'request_id': 'req-a60e88f9-d0e8-47cb-b004-adeab49999eb'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.343404 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:52.355345 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a60e88f9-d0e8-47cb-b004-adeab49999eb admin admin] The token cache is being invalidate because project 8fd848b22f854563a0ab63ec0971b595 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:52.355834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a60e88f9-d0e8-47cb-b004-adeab49999eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.356605 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a60e88f9-d0e8-47cb-b004-adeab49999eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.358350 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1102/2187] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:52 2022] DELETE /identity/v3/projects/8fd848b22f854563a0ab63ec0971b595 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:52.362973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da11dfe-6137-407d-9982-606730138ba9 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.363371 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da11dfe-6137-407d-9982-606730138ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.363557 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da11dfe-6137-407d-9982-606730138ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.383931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8da11dfe-6137-407d-9982-606730138ba9 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:52.385856 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8da11dfe-6137-407d-9982-606730138ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:52.446449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1086/2188] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:52.454982 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e0e2f-c88b-46eb-a675-50f29dd83b74 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.455274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e0e2f-c88b-46eb-a675-50f29dd83b74 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.455574 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e0e2f-c88b-46eb-a675-50f29dd83b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.504020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-6d7e0e2f-c88b-46eb-a675-50f29dd83b74 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:52.505275 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-6d7e0e2f-c88b-46eb-a675-50f29dd83b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:52.519592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1103/2189] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:52.523798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da11dfe-6137-407d-9982-606730138ba9 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.536904 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da11dfe-6137-407d-9982-606730138ba9 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.542011 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e0e2f-c88b-46eb-a675-50f29dd83b74 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.543715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce00e3-fbd9-4e31-a8ba-00cdf525ae40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.544517 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-6d7e0e2f-c88b-46eb-a675-50f29dd83b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:52.544655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce00e3-fbd9-4e31-a8ba-00cdf525ae40 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.544901 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce00e3-fbd9-4e31-a8ba-00cdf525ae40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.546660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce00e3-fbd9-4e31-a8ba-00cdf525ae40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.547418 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce00e3-fbd9-4e31-a8ba-00cdf525ae40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:52.547933 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce00e3-fbd9-4e31-a8ba-00cdf525ae40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.550815 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce00e3-fbd9-4e31-a8ba-00cdf525ae40 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.563720 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e0e2f-c88b-46eb-a675-50f29dd83b74 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.569548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf763b6-e88a-445e-858d-7da3f416b69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.570409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf763b6-e88a-445e-858d-7da3f416b69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.570875 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf763b6-e88a-445e-858d-7da3f416b69e admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.571332 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf763b6-e88a-445e-858d-7da3f416b69e admin admin] PATH_INFO: `/v3/projects/91d67d37f7cc4d869a36393d925c79cb` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.574850 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf763b6-e88a-445e-858d-7da3f416b69e admin admin] RBAC: Authorizing `identity:delete_project(project_id=91d67d37f7cc4d869a36393d925c79cb)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.576079 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1087/2190] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:52.577098 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce00e3-fbd9-4e31-a8ba-00cdf525ae40 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.578642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf763b6-e88a-445e-858d-7da3f416b69e admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.604608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce00e3-fbd9-4e31-a8ba-00cdf525ae40 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.613068 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12484ef1-64cb-4897-a415-9c787dcf20cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.613795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12484ef1-64cb-4897-a415-9c787dcf20cb admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.614145 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12484ef1-64cb-4897-a415-9c787dcf20cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.614486 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12484ef1-64cb-4897-a415-9c787dcf20cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.615200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12484ef1-64cb-4897-a415-9c787dcf20cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.617289 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12484ef1-64cb-4897-a415-9c787dcf20cb admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.648378 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5cf763b6-e88a-445e-858d-7da3f416b69e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91d67d37f7cc4d869a36393d925c79cb', 'request_id': 'req-5cf763b6-e88a-445e-858d-7da3f416b69e'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.663927 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1088/2191] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:52 2022] POST /identity/v3/projects => generated 367 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:52.665060 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12484ef1-64cb-4897-a415-9c787dcf20cb admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.668368 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5cf763b6-e88a-445e-858d-7da3f416b69e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91d67d37f7cc4d869a36393d925c79cb', 'request_id': 'req-5cf763b6-e88a-445e-858d-7da3f416b69e'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.683970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5cf763b6-e88a-445e-858d-7da3f416b69e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91d67d37f7cc4d869a36393d925c79cb', 'request_id': 'req-5cf763b6-e88a-445e-858d-7da3f416b69e'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.710138 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5cf763b6-e88a-445e-858d-7da3f416b69e admin admin] The token cache is being invalidate because project 91d67d37f7cc4d869a36393d925c79cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:52.712057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5cf763b6-e88a-445e-858d-7da3f416b69e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.714213 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5cf763b6-e88a-445e-858d-7da3f416b69e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.719942 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1104/2192] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:52 2022] DELETE /identity/v3/projects/91d67d37f7cc4d869a36393d925c79cb => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:52.721537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf763b6-e88a-445e-858d-7da3f416b69e admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.724517 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5cf763b6-e88a-445e-858d-7da3f416b69e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:52.770253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12484ef1-64cb-4897-a415-9c787dcf20cb admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.776258 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7dcea4-91af-4bba-b9db-e40b53058ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.777043 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dcea4-91af-4bba-b9db-e40b53058ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.777434 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dcea4-91af-4bba-b9db-e40b53058ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.777727 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dcea4-91af-4bba-b9db-e40b53058ee0 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.778375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7dcea4-91af-4bba-b9db-e40b53058ee0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.780690 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7dcea4-91af-4bba-b9db-e40b53058ee0 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.807402 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1089/2193] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:52 2022] POST /identity/v3/users => generated 284 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:24:52.808866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7dcea4-91af-4bba-b9db-e40b53058ee0 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.810976 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4f7dcea4-91af-4bba-b9db-e40b53058ee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:52.824825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7dcea4-91af-4bba-b9db-e40b53058ee0 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.829666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf763b6-e88a-445e-858d-7da3f416b69e admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:52.830461 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e335a2d-55dd-4b51-910a-703b3cb1be49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.831080 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e335a2d-55dd-4b51-910a-703b3cb1be49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.831360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e335a2d-55dd-4b51-910a-703b3cb1be49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.831631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e335a2d-55dd-4b51-910a-703b3cb1be49 admin admin] PATH_INFO: `/v3/projects/1218fa07fdaa4ca481f3f0b8f0847858` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.832757 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e335a2d-55dd-4b51-910a-703b3cb1be49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1218fa07fdaa4ca481f3f0b8f0847858)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.834934 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e335a2d-55dd-4b51-910a-703b3cb1be49 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.835191 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7457de-3e68-4929-a711-2dc20d2fed69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:52.836000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7457de-3e68-4929-a711-2dc20d2fed69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.836352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7457de-3e68-4929-a711-2dc20d2fed69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.836771 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7457de-3e68-4929-a711-2dc20d2fed69 admin admin] PATH_INFO: `/v3/users/9f3a9c1061394e03aab05ecdf19b0dd2` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.838024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7457de-3e68-4929-a711-2dc20d2fed69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f3a9c1061394e03aab05ecdf19b0dd2)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:52.840000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7457de-3e68-4929-a711-2dc20d2fed69 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:52.879578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:52.889597 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8d7457de-3e68-4929-a711-2dc20d2fed69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f3a9c1061394e03aab05ecdf19b0dd2'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.890264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8d7457de-3e68-4929-a711-2dc20d2fed69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f3a9c1061394e03aab05ecdf19b0dd2'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.890400 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6e335a2d-55dd-4b51-910a-703b3cb1be49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1218fa07fdaa4ca481f3f0b8f0847858', 'request_id': 'req-6e335a2d-55dd-4b51-910a-703b3cb1be49'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.890956 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8d7457de-3e68-4929-a711-2dc20d2fed69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f3a9c1061394e03aab05ecdf19b0dd2'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.897591 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6e335a2d-55dd-4b51-910a-703b3cb1be49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1218fa07fdaa4ca481f3f0b8f0847858', 'request_id': 'req-6e335a2d-55dd-4b51-910a-703b3cb1be49'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.898336 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8d7457de-3e68-4929-a711-2dc20d2fed69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f3a9c1061394e03aab05ecdf19b0dd2'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.904122 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6e335a2d-55dd-4b51-910a-703b3cb1be49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1218fa07fdaa4ca481f3f0b8f0847858', 'request_id': 'req-6e335a2d-55dd-4b51-910a-703b3cb1be49'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.911085 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8d7457de-3e68-4929-a711-2dc20d2fed69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f3a9c1061394e03aab05ecdf19b0dd2'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.916137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:52.917515 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6e335a2d-55dd-4b51-910a-703b3cb1be49 admin admin] The token cache is being invalidate because project 1218fa07fdaa4ca481f3f0b8f0847858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:52.917810 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6e335a2d-55dd-4b51-910a-703b3cb1be49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.918889 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6e335a2d-55dd-4b51-910a-703b3cb1be49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:52.920984 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1090/2194] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:52 2022] DELETE /identity/v3/projects/1218fa07fdaa4ca481f3f0b8f0847858 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:52.921693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e335a2d-55dd-4b51-910a-703b3cb1be49 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:52.923093 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6e335a2d-55dd-4b51-910a-703b3cb1be49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:52.932773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1105/2195] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:52 2022] DELETE /identity/v3/users/9f3a9c1061394e03aab05ecdf19b0dd2 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:52.936670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5ccca-5f81-46dc-b4dc-ba03c96edd23 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:52.936954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5ccca-5f81-46dc-b4dc-ba03c96edd23 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:52.937224 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5ccca-5f81-46dc-b4dc-ba03c96edd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:52.957691 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-3ac5ccca-5f81-46dc-b4dc-ba03c96edd23 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:52.959168 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3ac5ccca-5f81-46dc-b4dc-ba03c96edd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:52.997659 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e335a2d-55dd-4b51-910a-703b3cb1be49 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.035485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1106/2196] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:53.041079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201febc8-502b-4060-b184-dfc1af42b7b7 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.041397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201febc8-502b-4060-b184-dfc1af42b7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.041630 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201febc8-502b-4060-b184-dfc1af42b7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.064052 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-201febc8-502b-4060-b184-dfc1af42b7b7 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:53.065325 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-201febc8-502b-4060-b184-dfc1af42b7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:53.071531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b771b8-b8c0-4083-a807-107e46c286e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.073401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b771b8-b8c0-4083-a807-107e46c286e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.073686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b771b8-b8c0-4083-a807-107e46c286e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.073960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b771b8-b8c0-4083-a807-107e46c286e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.074648 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b771b8-b8c0-4083-a807-107e46c286e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:53.075257 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1107/2197] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:24:53.077041 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b771b8-b8c0-4083-a807-107e46c286e1 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:53.079205 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201febc8-502b-4060-b184-dfc1af42b7b7 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:53.090004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1091/2198] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:53.093817 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b771b8-b8c0-4083-a807-107e46c286e1 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:53.095413 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201febc8-502b-4060-b184-dfc1af42b7b7 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.100373 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b014ff46-7d40-41a4-b9c3-2a1c9a44a2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.100977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014ff46-7d40-41a4-b9c3-2a1c9a44a2e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.101252 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014ff46-7d40-41a4-b9c3-2a1c9a44a2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.101520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014ff46-7d40-41a4-b9c3-2a1c9a44a2e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.102297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b014ff46-7d40-41a4-b9c3-2a1c9a44a2e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:53.102676 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b014ff46-7d40-41a4-b9c3-2a1c9a44a2e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:53.104853 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b014ff46-7d40-41a4-b9c3-2a1c9a44a2e2 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:53.105325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b771b8-b8c0-4083-a807-107e46c286e1 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.110401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ddd6e-8ebd-4466-855b-e57241a1b671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.111164 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ddd6e-8ebd-4466-855b-e57241a1b671 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.111484 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ddd6e-8ebd-4466-855b-e57241a1b671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.111863 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ddd6e-8ebd-4466-855b-e57241a1b671 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.112578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ddd6e-8ebd-4466-855b-e57241a1b671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:53.114315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ddd6e-8ebd-4466-855b-e57241a1b671 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:53.117196 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1108/2199] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:24:53.121967 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b014ff46-7d40-41a4-b9c3-2a1c9a44a2e2 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:53.128004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1092/2200] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:24:53.131859 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ddd6e-8ebd-4466-855b-e57241a1b671 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:53.132549 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b014ff46-7d40-41a4-b9c3-2a1c9a44a2e2 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.137838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d34fed-cec5-43ad-be10-80ebf94e5149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.138516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d34fed-cec5-43ad-be10-80ebf94e5149 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.138872 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d34fed-cec5-43ad-be10-80ebf94e5149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.139238 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d34fed-cec5-43ad-be10-80ebf94e5149 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.139987 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d34fed-cec5-43ad-be10-80ebf94e5149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:53.141954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d34fed-cec5-43ad-be10-80ebf94e5149 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:53.143875 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ddd6e-8ebd-4466-855b-e57241a1b671 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.148732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866fb5a-5061-429c-a165-3eae96ba5f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.149539 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866fb5a-5061-429c-a165-3eae96ba5f95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.149884 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866fb5a-5061-429c-a165-3eae96ba5f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.150164 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866fb5a-5061-429c-a165-3eae96ba5f95 admin admin] PATH_INFO: `/v3/projects/4702732564084d6faedba47ed90bcb20/users/4ab849eaf8a746988ef812a1839f8a42/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.166544 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866fb5a-5061-429c-a165-3eae96ba5f95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4702732564084d6faedba47ed90bcb20, user_id=4ab849eaf8a746988ef812a1839f8a42, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:53.169930 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866fb5a-5061-429c-a165-3eae96ba5f95 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:53.181649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1109/2201] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:53 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:24:53.183151 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d34fed-cec5-43ad-be10-80ebf94e5149 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:53.188265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1093/2202] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:53 2022] PUT /identity/v3/projects/4702732564084d6faedba47ed90bcb20/users/4ab849eaf8a746988ef812a1839f8a42/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:24:53.198931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0004c-36a8-4960-ab08-59166e0d6c00 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.199306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0004c-36a8-4960-ab08-59166e0d6c00 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.199586 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0004c-36a8-4960-ab08-59166e0d6c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.232169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-85a0004c-36a8-4960-ab08-59166e0d6c00 None None] MFA Rules not processed for user `4ab849eaf8a746988ef812a1839f8a42`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:53.233404 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-85a0004c-36a8-4960-ab08-59166e0d6c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:53.256724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d34fed-cec5-43ad-be10-80ebf94e5149 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.262363 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:53.313833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1094/2203] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:53.314727 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0004c-36a8-4960-ab08-59166e0d6c00 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:53.316128 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-85a0004c-36a8-4960-ab08-59166e0d6c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:53.319685 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:53.327703 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a049a-e857-4b97-8cbe-e49d3b50d4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.329025 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a049a-e857-4b97-8cbe-e49d3b50d4f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.329534 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a049a-e857-4b97-8cbe-e49d3b50d4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.330000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a049a-e857-4b97-8cbe-e49d3b50d4f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.330875 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a049a-e857-4b97-8cbe-e49d3b50d4f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:53.332868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a049a-e857-4b97-8cbe-e49d3b50d4f8 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:53.355900 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1110/2204] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:53 2022] POST /identity/v3/users => generated 284 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:53.356960 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a049a-e857-4b97-8cbe-e49d3b50d4f8 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:53.358618 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b31a049a-e857-4b97-8cbe-e49d3b50d4f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:53.385881 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0004c-36a8-4960-ab08-59166e0d6c00 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.390674 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440afb4-7339-4413-915d-a6ee4dc4740f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.391305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440afb4-7339-4413-915d-a6ee4dc4740f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.391591 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440afb4-7339-4413-915d-a6ee4dc4740f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.391893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440afb4-7339-4413-915d-a6ee4dc4740f admin admin] PATH_INFO: `/v3/projects/2b390c6aadae4ab5b266e08a1fc6b7c5` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.393026 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440afb4-7339-4413-915d-a6ee4dc4740f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b390c6aadae4ab5b266e08a1fc6b7c5)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:53.394941 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440afb4-7339-4413-915d-a6ee4dc4740f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:53.418303 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a049a-e857-4b97-8cbe-e49d3b50d4f8 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.422110 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b64440-608a-421d-8032-935bb1b39034 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.422725 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b64440-608a-421d-8032-935bb1b39034 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.423003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b64440-608a-421d-8032-935bb1b39034 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.423315 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b64440-608a-421d-8032-935bb1b39034 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.432448 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b64440-608a-421d-8032-935bb1b39034 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:53.433902 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b64440-608a-421d-8032-935bb1b39034 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:53.443229 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1111/2205] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:53.443529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-2440afb4-7339-4413-915d-a6ee4dc4740f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b390c6aadae4ab5b266e08a1fc6b7c5', 'request_id': 'req-2440afb4-7339-4413-915d-a6ee4dc4740f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:53.444121 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b64440-608a-421d-8032-935bb1b39034 service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:53.450765 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-2440afb4-7339-4413-915d-a6ee4dc4740f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b390c6aadae4ab5b266e08a1fc6b7c5', 'request_id': 'req-2440afb4-7339-4413-915d-a6ee4dc4740f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:53.457136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-2440afb4-7339-4413-915d-a6ee4dc4740f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b390c6aadae4ab5b266e08a1fc6b7c5', 'request_id': 'req-2440afb4-7339-4413-915d-a6ee4dc4740f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:53.457519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b64440-608a-421d-8032-935bb1b39034 service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.473173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-2440afb4-7339-4413-915d-a6ee4dc4740f admin admin] The token cache is being invalidate because project 2b390c6aadae4ab5b266e08a1fc6b7c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:53.473563 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-2440afb4-7339-4413-915d-a6ee4dc4740f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:53.474311 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-2440afb4-7339-4413-915d-a6ee4dc4740f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:53.476082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1095/2206] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:53 2022] DELETE /identity/v3/projects/2b390c6aadae4ab5b266e08a1fc6b7c5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:53.480744 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709d47c-21e8-4e9e-8f18-dc33b05aca0e None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.481305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709d47c-21e8-4e9e-8f18-dc33b05aca0e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.481882 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709d47c-21e8-4e9e-8f18-dc33b05aca0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.503421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f709d47c-21e8-4e9e-8f18-dc33b05aca0e None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:53.504688 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f709d47c-21e8-4e9e-8f18-dc33b05aca0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:53.523126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887a7b0-2ec1-442d-948f-b973ff0f81c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.523729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887a7b0-2ec1-442d-948f-b973ff0f81c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.524045 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887a7b0-2ec1-442d-948f-b973ff0f81c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.524323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887a7b0-2ec1-442d-948f-b973ff0f81c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.525606 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887a7b0-2ec1-442d-948f-b973ff0f81c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:53.527215 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887a7b0-2ec1-442d-948f-b973ff0f81c4 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:53.537550 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1112/2207] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:53.541371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887a7b0-2ec1-442d-948f-b973ff0f81c4 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:53.543927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3887a7b0-2ec1-442d-948f-b973ff0f81c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:53.557350 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887a7b0-2ec1-442d-948f-b973ff0f81c4 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.560157 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:53.563225 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a58b75-d777-426e-9026-21ab63713b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.564026 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a58b75-d777-426e-9026-21ab63713b9e admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.564240 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a58b75-d777-426e-9026-21ab63713b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.564514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a58b75-d777-426e-9026-21ab63713b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.565159 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a58b75-d777-426e-9026-21ab63713b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:53.566755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a58b75-d777-426e-9026-21ab63713b9e admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:53.577792 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1096/2208] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:24:53.578913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1113/2209] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:24:53.586355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709d47c-21e8-4e9e-8f18-dc33b05aca0e None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:53.589226 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef24d8b-2026-4ac7-b1dc-00683bee18da None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.589518 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef24d8b-2026-4ac7-b1dc-00683bee18da None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.589798 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef24d8b-2026-4ac7-b1dc-00683bee18da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.597399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709d47c-21e8-4e9e-8f18-dc33b05aca0e None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.603676 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e418da4-a4de-44e6-bf88-da6afd5af641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.604602 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e418da4-a4de-44e6-bf88-da6afd5af641 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.604792 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e418da4-a4de-44e6-bf88-da6afd5af641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.605099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e418da4-a4de-44e6-bf88-da6afd5af641 admin admin] PATH_INFO: `/v3/projects/0efa2de623ef4eea8055263ced591b55/users/425ad4f89595428e816ce005de4c6241/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.615178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-7ef24d8b-2026-4ac7-b1dc-00683bee18da None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:53.616539 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-7ef24d8b-2026-4ac7-b1dc-00683bee18da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:53.623233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e418da4-a4de-44e6-bf88-da6afd5af641 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0efa2de623ef4eea8055263ced591b55, user_id=425ad4f89595428e816ce005de4c6241, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:53.626527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e418da4-a4de-44e6-bf88-da6afd5af641 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:53.630857 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1114/2210] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:53.631864 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef24d8b-2026-4ac7-b1dc-00683bee18da None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:53.642044 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef24d8b-2026-4ac7-b1dc-00683bee18da None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.646034 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1097/2211] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:53 2022] PUT /identity/v3/projects/0efa2de623ef4eea8055263ced591b55/users/425ad4f89595428e816ce005de4c6241/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:53.657428 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2950eb-1c63-48cd-b81d-8470c0234933 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.657713 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2950eb-1c63-48cd-b81d-8470c0234933 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.657984 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2950eb-1c63-48cd-b81d-8470c0234933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.704739 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-ad2950eb-1c63-48cd-b81d-8470c0234933 None None] MFA Rules not processed for user `425ad4f89595428e816ce005de4c6241`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:53.706565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ad2950eb-1c63-48cd-b81d-8470c0234933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:53.738677 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ddc18-15fa-497a-8d98-84c3735c5ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.739286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ddc18-15fa-497a-8d98-84c3735c5ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.740338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ddc18-15fa-497a-8d98-84c3735c5ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.740566 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ddc18-15fa-497a-8d98-84c3735c5ce5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.741226 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ddc18-15fa-497a-8d98-84c3735c5ce5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:53.741860 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ddc18-15fa-497a-8d98-84c3735c5ce5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:53.743566 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ddc18-15fa-497a-8d98-84c3735c5ce5 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:53.777010 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1115/2212] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:53.779894 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ddc18-15fa-497a-8d98-84c3735c5ce5 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:53.787802 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:53.797361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ddc18-15fa-497a-8d98-84c3735c5ce5 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.806753 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf587e5-fdf8-4716-9d22-be145e6d1468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.811328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf587e5-fdf8-4716-9d22-be145e6d1468 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.812546 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf587e5-fdf8-4716-9d22-be145e6d1468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.812824 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf587e5-fdf8-4716-9d22-be145e6d1468 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.813547 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf587e5-fdf8-4716-9d22-be145e6d1468 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:53.814457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1098/2213] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:53.816766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf587e5-fdf8-4716-9d22-be145e6d1468 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:53.828031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2950eb-1c63-48cd-b81d-8470c0234933 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:53.829822 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ad2950eb-1c63-48cd-b81d-8470c0234933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:53.888871 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1116/2214] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:53 2022] POST /identity/v3/projects => generated 369 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:53.893346 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf587e5-fdf8-4716-9d22-be145e6d1468 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:53.924657 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2950eb-1c63-48cd-b81d-8470c0234933 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.987361 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387ec3f-4edd-440a-8d86-85a95d549eb9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88670b0e1a0f460d8f74917c92db9b9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.988731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387ec3f-4edd-440a-8d86-85a95d549eb9 service glance] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:53.989019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387ec3f-4edd-440a-8d86-85a95d549eb9 service glance] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:53.989296 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387ec3f-4edd-440a-8d86-85a95d549eb9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:53.993238 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf587e5-fdf8-4716-9d22-be145e6d1468 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:53.999152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17614cbc-7412-4601-b750-dbe25603c062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:53.999629 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17614cbc-7412-4601-b750-dbe25603c062 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:54.000064 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17614cbc-7412-4601-b750-dbe25603c062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:54.000337 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17614cbc-7412-4601-b750-dbe25603c062 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:54.000975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17614cbc-7412-4601-b750-dbe25603c062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:54.003570 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17614cbc-7412-4601-b750-dbe25603c062 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:54.024999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:54.029987 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1117/2215] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:53 2022] POST /identity/v3/users => generated 285 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:54.033873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17614cbc-7412-4601-b750-dbe25603c062 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:54.045608 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17614cbc-7412-4601-b750-dbe25603c062 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:54.050796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ab960-34cf-418a-96e4-111007dae92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:54.051414 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ab960-34cf-418a-96e4-111007dae92b admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:54.051686 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ab960-34cf-418a-96e4-111007dae92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:54.051995 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ab960-34cf-418a-96e4-111007dae92b admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:54.052684 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3ab960-34cf-418a-96e4-111007dae92b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:54.054287 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3ab960-34cf-418a-96e4-111007dae92b admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:54.059772 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387ec3f-4edd-440a-8d86-85a95d549eb9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:54.061239 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387ec3f-4edd-440a-8d86-85a95d549eb9 service glance] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:54.066421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1118/2216] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:24:54.070252 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ab960-34cf-418a-96e4-111007dae92b admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:54.071843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1099/2217] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:54.080105 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ab960-34cf-418a-96e4-111007dae92b admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:54.085220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3142b6-e27d-47f6-99a3-a5751fd84c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:54.085873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3142b6-e27d-47f6-99a3-a5751fd84c10 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:54.086157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3142b6-e27d-47f6-99a3-a5751fd84c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:54.086541 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3142b6-e27d-47f6-99a3-a5751fd84c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:54.087202 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3142b6-e27d-47f6-99a3-a5751fd84c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:54.088784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3142b6-e27d-47f6-99a3-a5751fd84c10 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:54.103341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1119/2218] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:24:54.104444 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387ec3f-4edd-440a-8d86-85a95d549eb9 service glance] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:54.115331 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387ec3f-4edd-440a-8d86-85a95d549eb9 service glance] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:54.120847 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd88d4-ca86-4798-b0d7-47ce11435d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:54.121492 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd88d4-ca86-4798-b0d7-47ce11435d7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:54.121774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd88d4-ca86-4798-b0d7-47ce11435d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:54.122052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd88d4-ca86-4798-b0d7-47ce11435d7e admin admin] PATH_INFO: `/v3/projects/65942cc2473749c0acd96430ec0295a5/users/d02c8eb38b6c4b5f9cfea5c766f69bfb/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:54.137804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd88d4-ca86-4798-b0d7-47ce11435d7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=65942cc2473749c0acd96430ec0295a5, user_id=d02c8eb38b6c4b5f9cfea5c766f69bfb, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:54.141052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd88d4-ca86-4798-b0d7-47ce11435d7e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:54.167840 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1100/2219] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:54 2022] PUT /identity/v3/projects/65942cc2473749c0acd96430ec0295a5/users/d02c8eb38b6c4b5f9cfea5c766f69bfb/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:54.169773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1e429-29df-41f2-bbbf-e012e95a7468 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:54.170317 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1e429-29df-41f2-bbbf-e012e95a7468 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:54.170394 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1e429-29df-41f2-bbbf-e012e95a7468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:54.199408 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-e1d1e429-29df-41f2-bbbf-e012e95a7468 None None] MFA Rules not processed for user `d02c8eb38b6c4b5f9cfea5c766f69bfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:54.200707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e1d1e429-29df-41f2-bbbf-e012e95a7468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:54.265951 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1120/2220] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:54.278244 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd88d4-ca86-4798-b0d7-47ce11435d7e admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:54.280628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-58fd88d4-ca86-4798-b0d7-47ce11435d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:54.341057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd88d4-ca86-4798-b0d7-47ce11435d7e admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:54.344706 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa5191-40a6-4ec0-b48e-c4c30e18f534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:54.345318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa5191-40a6-4ec0-b48e-c4c30e18f534 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:54.345571 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa5191-40a6-4ec0-b48e-c4c30e18f534 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:54.345864 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa5191-40a6-4ec0-b48e-c4c30e18f534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:54.354928 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa5191-40a6-4ec0-b48e-c4c30e18f534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:54.356567 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa5191-40a6-4ec0-b48e-c4c30e18f534 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:54.366301 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1101/2221] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:24:54.393570 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1e429-29df-41f2-bbbf-e012e95a7468 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:54.437220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:54.461492 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1e429-29df-41f2-bbbf-e012e95a7468 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:54.466418 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cdea28-8f3f-46e5-81a1-169b95cc4cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:54.467039 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cdea28-8f3f-46e5-81a1-169b95cc4cf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:54.467403 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cdea28-8f3f-46e5-81a1-169b95cc4cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:54.467567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cdea28-8f3f-46e5-81a1-169b95cc4cf9 admin admin] PATH_INFO: `/v3/users/d02c8eb38b6c4b5f9cfea5c766f69bfb` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:54.468735 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cdea28-8f3f-46e5-81a1-169b95cc4cf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d02c8eb38b6c4b5f9cfea5c766f69bfb)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:54.470659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cdea28-8f3f-46e5-81a1-169b95cc4cf9 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:54.534221 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-46cdea28-8f3f-46e5-81a1-169b95cc4cf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd02c8eb38b6c4b5f9cfea5c766f69bfb'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:54.535598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-46cdea28-8f3f-46e5-81a1-169b95cc4cf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd02c8eb38b6c4b5f9cfea5c766f69bfb'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:54.536257 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-46cdea28-8f3f-46e5-81a1-169b95cc4cf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd02c8eb38b6c4b5f9cfea5c766f69bfb'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:54.544113 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-46cdea28-8f3f-46e5-81a1-169b95cc4cf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd02c8eb38b6c4b5f9cfea5c766f69bfb'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:54.558247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-46cdea28-8f3f-46e5-81a1-169b95cc4cf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd02c8eb38b6c4b5f9cfea5c766f69bfb'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:54.572439 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1121/2222] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:54 2022] DELETE /identity/v3/users/d02c8eb38b6c4b5f9cfea5c766f69bfb => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:54.576825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2c565-82d3-4161-89c1-67857c3604a7 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:54.577313 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2c565-82d3-4161-89c1-67857c3604a7 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:54.577404 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2c565-82d3-4161-89c1-67857c3604a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:54.597176 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-0ba2c565-82d3-4161-89c1-67857c3604a7 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:54.598373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0ba2c565-82d3-4161-89c1-67857c3604a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:54.656870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1102/2223] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:54.666816 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cdea28-8f3f-46e5-81a1-169b95cc4cf9 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:54.668303 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-46cdea28-8f3f-46e5-81a1-169b95cc4cf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:54.725375 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cdea28-8f3f-46e5-81a1-169b95cc4cf9 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:54.729036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41805780-c3f5-4c6b-9cbd-83b2612dec20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:54.729636 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41805780-c3f5-4c6b-9cbd-83b2612dec20 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:54.729915 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41805780-c3f5-4c6b-9cbd-83b2612dec20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:54.730187 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41805780-c3f5-4c6b-9cbd-83b2612dec20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:54.739568 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41805780-c3f5-4c6b-9cbd-83b2612dec20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:54.741035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41805780-c3f5-4c6b-9cbd-83b2612dec20 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:54.751299 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1122/2224] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:55.051455 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2c565-82d3-4161-89c1-67857c3604a7 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:55.053029 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0ba2c565-82d3-4161-89c1-67857c3604a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:55.067841 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2c565-82d3-4161-89c1-67857c3604a7 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:55.072977 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9610a5-a7fd-4389-a4d9-2f8109f33fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:55.073626 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9610a5-a7fd-4389-a4d9-2f8109f33fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:55.073979 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9610a5-a7fd-4389-a4d9-2f8109f33fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:55.074176 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9610a5-a7fd-4389-a4d9-2f8109f33fc0 admin admin] PATH_INFO: `/v3/projects/65942cc2473749c0acd96430ec0295a5` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:55.075323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9610a5-a7fd-4389-a4d9-2f8109f33fc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65942cc2473749c0acd96430ec0295a5)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:55.077282 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9610a5-a7fd-4389-a4d9-2f8109f33fc0 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:55.131261 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:55.137417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6d9610a5-a7fd-4389-a4d9-2f8109f33fc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65942cc2473749c0acd96430ec0295a5', 'request_id': 'req-6d9610a5-a7fd-4389-a4d9-2f8109f33fc0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:55.147996 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6d9610a5-a7fd-4389-a4d9-2f8109f33fc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65942cc2473749c0acd96430ec0295a5', 'request_id': 'req-6d9610a5-a7fd-4389-a4d9-2f8109f33fc0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:55.155788 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6d9610a5-a7fd-4389-a4d9-2f8109f33fc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65942cc2473749c0acd96430ec0295a5', 'request_id': 'req-6d9610a5-a7fd-4389-a4d9-2f8109f33fc0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:55.173634 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6d9610a5-a7fd-4389-a4d9-2f8109f33fc0 admin admin] The token cache is being invalidate because project 65942cc2473749c0acd96430ec0295a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:55.173941 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6d9610a5-a7fd-4389-a4d9-2f8109f33fc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:55.174585 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6d9610a5-a7fd-4389-a4d9-2f8109f33fc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:55.176336 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1103/2225] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:55 2022] DELETE /identity/v3/projects/65942cc2473749c0acd96430ec0295a5 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:57.340258 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41805780-c3f5-4c6b-9cbd-83b2612dec20 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:57.341720 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-41805780-c3f5-4c6b-9cbd-83b2612dec20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:57.355934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41805780-c3f5-4c6b-9cbd-83b2612dec20 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:57.415363 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d6528-23f6-4e33-a9a4-bc75632d796e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:57.416030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d6528-23f6-4e33-a9a4-bc75632d796e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:57.416313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d6528-23f6-4e33-a9a4-bc75632d796e admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:57.416586 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d6528-23f6-4e33-a9a4-bc75632d796e admin admin] PATH_INFO: `/v3/users/3dd26ea6bdee48a8ab19dbd157083b5b` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:57.417724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d6528-23f6-4e33-a9a4-bc75632d796e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dd26ea6bdee48a8ab19dbd157083b5b)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:57.419625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d6528-23f6-4e33-a9a4-bc75632d796e admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:57.421154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:57.469308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ea3d6528-23f6-4e33-a9a4-bc75632d796e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dd26ea6bdee48a8ab19dbd157083b5b'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:57.469961 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ea3d6528-23f6-4e33-a9a4-bc75632d796e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dd26ea6bdee48a8ab19dbd157083b5b'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:57.470556 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ea3d6528-23f6-4e33-a9a4-bc75632d796e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dd26ea6bdee48a8ab19dbd157083b5b'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:57.477638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ea3d6528-23f6-4e33-a9a4-bc75632d796e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dd26ea6bdee48a8ab19dbd157083b5b'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:57.489162 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ea3d6528-23f6-4e33-a9a4-bc75632d796e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dd26ea6bdee48a8ab19dbd157083b5b'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:57.503045 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1123/2226] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:57 2022] DELETE /identity/v3/users/3dd26ea6bdee48a8ab19dbd157083b5b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:57.507853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94632cf-6424-4742-91a6-fac267742b69 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:57.508337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94632cf-6424-4742-91a6-fac267742b69 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:57.508432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94632cf-6424-4742-91a6-fac267742b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:57.528710 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-b94632cf-6424-4742-91a6-fac267742b69 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:57.529950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b94632cf-6424-4742-91a6-fac267742b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:57.589284 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1104/2227] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:57.598879 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d6528-23f6-4e33-a9a4-bc75632d796e admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:57.600307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ea3d6528-23f6-4e33-a9a4-bc75632d796e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:57.659998 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d6528-23f6-4e33-a9a4-bc75632d796e admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:57.664049 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101b369-e42d-481a-8b5e-42ff3dd2ccf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:57.664666 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101b369-e42d-481a-8b5e-42ff3dd2ccf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:57.664947 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101b369-e42d-481a-8b5e-42ff3dd2ccf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:57.665223 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101b369-e42d-481a-8b5e-42ff3dd2ccf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:57.675022 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101b369-e42d-481a-8b5e-42ff3dd2ccf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:57.676485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101b369-e42d-481a-8b5e-42ff3dd2ccf0 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:57.686751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1124/2228] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:57.968283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94632cf-6424-4742-91a6-fac267742b69 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:57.969792 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b94632cf-6424-4742-91a6-fac267742b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:57.984401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94632cf-6424-4742-91a6-fac267742b69 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:58.003090 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033313bb-1e52-414f-a8e0-4c0593c7e306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:58.003831 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033313bb-1e52-414f-a8e0-4c0593c7e306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:58.004132 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033313bb-1e52-414f-a8e0-4c0593c7e306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:58.004401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033313bb-1e52-414f-a8e0-4c0593c7e306 admin admin] PATH_INFO: `/v3/projects/80e8b3a36fc641d5b0eb5968394a108d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:58.005848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033313bb-1e52-414f-a8e0-4c0593c7e306 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80e8b3a36fc641d5b0eb5968394a108d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:58.007789 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033313bb-1e52-414f-a8e0-4c0593c7e306 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:58.066398 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-033313bb-1e52-414f-a8e0-4c0593c7e306 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80e8b3a36fc641d5b0eb5968394a108d', 'request_id': 'req-033313bb-1e52-414f-a8e0-4c0593c7e306'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:58.074110 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-033313bb-1e52-414f-a8e0-4c0593c7e306 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80e8b3a36fc641d5b0eb5968394a108d', 'request_id': 'req-033313bb-1e52-414f-a8e0-4c0593c7e306'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:58.081716 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-033313bb-1e52-414f-a8e0-4c0593c7e306 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80e8b3a36fc641d5b0eb5968394a108d', 'request_id': 'req-033313bb-1e52-414f-a8e0-4c0593c7e306'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:58.098021 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-033313bb-1e52-414f-a8e0-4c0593c7e306 admin admin] The token cache is being invalidate because project 80e8b3a36fc641d5b0eb5968394a108d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:24:58.098323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-033313bb-1e52-414f-a8e0-4c0593c7e306 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:58.099197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-033313bb-1e52-414f-a8e0-4c0593c7e306 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:58.101660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1105/2229] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:57 2022] DELETE /identity/v3/projects/80e8b3a36fc641d5b0eb5968394a108d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:58.106515 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5090f-39e5-4c27-9156-c19b7cbfb799 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:58.106981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5090f-39e5-4c27-9156-c19b7cbfb799 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:58.107469 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5090f-39e5-4c27-9156-c19b7cbfb799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:58.127914 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-72f5090f-39e5-4c27-9156-c19b7cbfb799 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:58.129205 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-72f5090f-39e5-4c27-9156-c19b7cbfb799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:58.206916 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1125/2230] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:24:58.216589 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831c745-3865-420a-926a-df6ad6642fb7 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:58.217069 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831c745-3865-420a-926a-df6ad6642fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:58.217619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831c745-3865-420a-926a-df6ad6642fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:58.260107 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-0831c745-3865-420a-926a-df6ad6642fb7 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:58.261572 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0831c745-3865-420a-926a-df6ad6642fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:58.271834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1106/2231] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:58.276207 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5090f-39e5-4c27-9156-c19b7cbfb799 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:58.286818 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5090f-39e5-4c27-9156-c19b7cbfb799 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:58.292457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3feaa3-725f-403e-ba80-f070067fa209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:58.293068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3feaa3-725f-403e-ba80-f070067fa209 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:58.293595 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3feaa3-725f-403e-ba80-f070067fa209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:58.293893 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3feaa3-725f-403e-ba80-f070067fa209 admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:58.299299 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3feaa3-725f-403e-ba80-f070067fa209 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:24:58.299669 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3feaa3-725f-403e-ba80-f070067fa209 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:58.301497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3feaa3-725f-403e-ba80-f070067fa209 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:58.319973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1126/2232] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:24:58.320945 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831c745-3865-420a-926a-df6ad6642fb7 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:58.331140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831c745-3865-420a-926a-df6ad6642fb7 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:58.336348 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951778e6-fce7-4810-b6e0-7f5514afbf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:58.337468 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951778e6-fce7-4810-b6e0-7f5514afbf36 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:58.337869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951778e6-fce7-4810-b6e0-7f5514afbf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:58.338154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951778e6-fce7-4810-b6e0-7f5514afbf36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:58.338797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951778e6-fce7-4810-b6e0-7f5514afbf36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:58.340772 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951778e6-fce7-4810-b6e0-7f5514afbf36 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:58.381488 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3feaa3-725f-403e-ba80-f070067fa209 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:58.383950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1107/2233] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:24:58 2022] POST /identity/v3/projects => generated 359 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:24:58.393584 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3feaa3-725f-403e-ba80-f070067fa209 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:58.398845 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:58.479488 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58a616-d427-411e-a4f4-9ac7ac728cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:58.480995 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58a616-d427-411e-a4f4-9ac7ac728cdf admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:58.481274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58a616-d427-411e-a4f4-9ac7ac728cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:58.481545 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58a616-d427-411e-a4f4-9ac7ac728cdf admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:58.482183 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58a616-d427-411e-a4f4-9ac7ac728cdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:58.484208 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58a616-d427-411e-a4f4-9ac7ac728cdf admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:58.524460 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1127/2234] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:24:58 2022] POST /identity/v3/users => generated 280 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:24:58.526450 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951778e6-fce7-4810-b6e0-7f5514afbf36 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:58.538145 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951778e6-fce7-4810-b6e0-7f5514afbf36 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:58.542935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc6cdd-b632-498e-9c21-7c1492b8199f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:58.552156 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc6cdd-b632-498e-9c21-7c1492b8199f admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:58.552451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc6cdd-b632-498e-9c21-7c1492b8199f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:58.552733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc6cdd-b632-498e-9c21-7c1492b8199f admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:58.553397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc6cdd-b632-498e-9c21-7c1492b8199f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:58.555028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc6cdd-b632-498e-9c21-7c1492b8199f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:58.567692 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1108/2235] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:58.570863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58a616-d427-411e-a4f4-9ac7ac728cdf admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:58.582744 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58a616-d427-411e-a4f4-9ac7ac728cdf admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:58.588540 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53397687-6933-4e05-9166-06576c4f896a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:58.589323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53397687-6933-4e05-9166-06576c4f896a admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:58.589677 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53397687-6933-4e05-9166-06576c4f896a admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:58.590029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53397687-6933-4e05-9166-06576c4f896a admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:58.590745 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53397687-6933-4e05-9166-06576c4f896a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:58.592460 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53397687-6933-4e05-9166-06576c4f896a admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:58.603975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1128/2236] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:24:58.608243 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc6cdd-b632-498e-9c21-7c1492b8199f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:58.620844 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc6cdd-b632-498e-9c21-7c1492b8199f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:58.625978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8a35a-9288-4a8b-bd24-4227c2563c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:58.626638 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8a35a-9288-4a8b-bd24-4227c2563c33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:58.626926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8a35a-9288-4a8b-bd24-4227c2563c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:58.627206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8a35a-9288-4a8b-bd24-4227c2563c33 admin admin] PATH_INFO: `/v3/projects/d28548b495d942959ab861d42dd8283e/users/350e6c6d2fc448008d16a1f56136cd97/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:58.641065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:58.646283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8a35a-9288-4a8b-bd24-4227c2563c33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d28548b495d942959ab861d42dd8283e, user_id=350e6c6d2fc448008d16a1f56136cd97, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:58.649178 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8a35a-9288-4a8b-bd24-4227c2563c33 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:58.686027 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1109/2237] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:24:58 2022] PUT /identity/v3/projects/d28548b495d942959ab861d42dd8283e/users/350e6c6d2fc448008d16a1f56136cd97/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:58.697908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb272b42-dc8a-434d-91ee-c670ac57a6eb None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:58.698356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb272b42-dc8a-434d-91ee-c670ac57a6eb None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:58.698716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb272b42-dc8a-434d-91ee-c670ac57a6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:58.731142 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-cb272b42-dc8a-434d-91ee-c670ac57a6eb None None] MFA Rules not processed for user `350e6c6d2fc448008d16a1f56136cd97`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:58.732507 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-cb272b42-dc8a-434d-91ee-c670ac57a6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:58.841431 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1129/2238] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:58 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:24:58.882668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8a35a-9288-4a8b-bd24-4227c2563c33 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:58.885506 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-85d8a35a-9288-4a8b-bd24-4227c2563c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:58.962388 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8a35a-9288-4a8b-bd24-4227c2563c33 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:58.966424 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bdafe5-1c5a-44c4-a64a-7e36722e02da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:58.967038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bdafe5-1c5a-44c4-a64a-7e36722e02da service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:58.967319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bdafe5-1c5a-44c4-a64a-7e36722e02da service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:58.967595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bdafe5-1c5a-44c4-a64a-7e36722e02da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:58.979062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bdafe5-1c5a-44c4-a64a-7e36722e02da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:58.981190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bdafe5-1c5a-44c4-a64a-7e36722e02da service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:58.991899 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1110/2239] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:58 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:59.017877 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb272b42-dc8a-434d-91ee-c670ac57a6eb None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:59.083105 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb272b42-dc8a-434d-91ee-c670ac57a6eb None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:59.087941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e721615-7388-44d2-b153-bc4f9bda50ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:59.088574 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e721615-7388-44d2-b153-bc4f9bda50ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:59.088855 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e721615-7388-44d2-b153-bc4f9bda50ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:59.089194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e721615-7388-44d2-b153-bc4f9bda50ac admin admin] PATH_INFO: `/v3/users/350e6c6d2fc448008d16a1f56136cd97` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:59.090356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e721615-7388-44d2-b153-bc4f9bda50ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=350e6c6d2fc448008d16a1f56136cd97)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:59.092324 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e721615-7388-44d2-b153-bc4f9bda50ac admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:59.113220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:24:59.140076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5e721615-7388-44d2-b153-bc4f9bda50ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '350e6c6d2fc448008d16a1f56136cd97'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:59.140793 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5e721615-7388-44d2-b153-bc4f9bda50ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '350e6c6d2fc448008d16a1f56136cd97'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:59.141403 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5e721615-7388-44d2-b153-bc4f9bda50ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '350e6c6d2fc448008d16a1f56136cd97'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:59.148511 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5e721615-7388-44d2-b153-bc4f9bda50ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '350e6c6d2fc448008d16a1f56136cd97'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:59.161099 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5e721615-7388-44d2-b153-bc4f9bda50ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '350e6c6d2fc448008d16a1f56136cd97'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:24:59.174462 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1130/2240] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:24:59 2022] DELETE /identity/v3/users/350e6c6d2fc448008d16a1f56136cd97 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:24:59.178989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d82d46-f823-4533-b9c1-70f08f573274 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:59.179306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d82d46-f823-4533-b9c1-70f08f573274 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:59.179584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d82d46-f823-4533-b9c1-70f08f573274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:59.199869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-08d82d46-f823-4533-b9c1-70f08f573274 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:24:59.201083 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-08d82d46-f823-4533-b9c1-70f08f573274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:59.261940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1111/2241] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:24:59.271838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e721615-7388-44d2-b153-bc4f9bda50ac admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:59.273290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5e721615-7388-44d2-b153-bc4f9bda50ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:59.333223 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e721615-7388-44d2-b153-bc4f9bda50ac admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:59.337048 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2756426-dd4c-4477-b95e-bb7b7a5e195e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:59.337648 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2756426-dd4c-4477-b95e-bb7b7a5e195e service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:59.337929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2756426-dd4c-4477-b95e-bb7b7a5e195e service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:59.338201 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2756426-dd4c-4477-b95e-bb7b7a5e195e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:59.347343 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2756426-dd4c-4477-b95e-bb7b7a5e195e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:59.348911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2756426-dd4c-4477-b95e-bb7b7a5e195e service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:24:59.361227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1131/2242] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:24:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:24:59.942668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d82d46-f823-4533-b9c1-70f08f573274 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:24:59.944141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-08d82d46-f823-4533-b9c1-70f08f573274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:24:59.955952 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d82d46-f823-4533-b9c1-70f08f573274 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:24:59.961038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5f6a0-0863-48dd-a04c-5b58bb68f1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:24:59.961694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5f6a0-0863-48dd-a04c-5b58bb68f1d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:24:59.962480 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5f6a0-0863-48dd-a04c-5b58bb68f1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:24:59.962755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5f6a0-0863-48dd-a04c-5b58bb68f1d2 admin admin] PATH_INFO: `/v3/projects/d28548b495d942959ab861d42dd8283e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:24:59.963897 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f5f6a0-0863-48dd-a04c-5b58bb68f1d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d28548b495d942959ab861d42dd8283e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:24:59.965861 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f5f6a0-0863-48dd-a04c-5b58bb68f1d2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:00.014539 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d7f5f6a0-0863-48dd-a04c-5b58bb68f1d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd28548b495d942959ab861d42dd8283e', 'request_id': 'req-d7f5f6a0-0863-48dd-a04c-5b58bb68f1d2'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:00.021101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d7f5f6a0-0863-48dd-a04c-5b58bb68f1d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd28548b495d942959ab861d42dd8283e', 'request_id': 'req-d7f5f6a0-0863-48dd-a04c-5b58bb68f1d2'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:00.027511 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d7f5f6a0-0863-48dd-a04c-5b58bb68f1d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd28548b495d942959ab861d42dd8283e', 'request_id': 'req-d7f5f6a0-0863-48dd-a04c-5b58bb68f1d2'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:00.046540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d7f5f6a0-0863-48dd-a04c-5b58bb68f1d2 admin admin] The token cache is being invalidate because project d28548b495d942959ab861d42dd8283e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:00.046835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d7f5f6a0-0863-48dd-a04c-5b58bb68f1d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:00.047482 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-d7f5f6a0-0863-48dd-a04c-5b58bb68f1d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:00.048992 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1112/2243] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:24:59 2022] DELETE /identity/v3/projects/d28548b495d942959ab861d42dd8283e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:00.053975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efba8af-8db8-46a2-b8d7-c6420a6baf4a None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:00.054438 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efba8af-8db8-46a2-b8d7-c6420a6baf4a None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:00.054543 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efba8af-8db8-46a2-b8d7-c6420a6baf4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:00.076915 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-5efba8af-8db8-46a2-b8d7-c6420a6baf4a None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:00.078256 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5efba8af-8db8-46a2-b8d7-c6420a6baf4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:00.152744 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1132/2244] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:25:00.160451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a750cb-39ed-4712-bb20-7935afd3ad32 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:00.160742 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a750cb-39ed-4712-bb20-7935afd3ad32 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:00.161019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a750cb-39ed-4712-bb20-7935afd3ad32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:00.180772 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-81a750cb-39ed-4712-bb20-7935afd3ad32 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:00.181976 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-81a750cb-39ed-4712-bb20-7935afd3ad32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:00.190790 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1113/2245] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:00.195711 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efba8af-8db8-46a2-b8d7-c6420a6baf4a None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:00.206189 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efba8af-8db8-46a2-b8d7-c6420a6baf4a None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:00.211179 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55d6ba-82f4-483b-8c70-736dda5d804a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:00.211800 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55d6ba-82f4-483b-8c70-736dda5d804a admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:00.212685 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55d6ba-82f4-483b-8c70-736dda5d804a admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:00.212955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55d6ba-82f4-483b-8c70-736dda5d804a admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:00.213590 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55d6ba-82f4-483b-8c70-736dda5d804a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:25:00.213891 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55d6ba-82f4-483b-8c70-736dda5d804a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:00.215467 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55d6ba-82f4-483b-8c70-736dda5d804a admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:00.227326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1133/2246] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:25:00.232200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a750cb-39ed-4712-bb20-7935afd3ad32 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:00.241319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a750cb-39ed-4712-bb20-7935afd3ad32 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:00.246533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68229a-ea59-4026-8437-4613f00c236b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:00.247115 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68229a-ea59-4026-8437-4613f00c236b admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:00.247390 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68229a-ea59-4026-8437-4613f00c236b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:00.247656 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68229a-ea59-4026-8437-4613f00c236b admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:00.248307 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68229a-ea59-4026-8437-4613f00c236b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:00.250206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68229a-ea59-4026-8437-4613f00c236b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:00.262663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:00.282043 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1114/2247] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:00 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:00.286691 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55d6ba-82f4-483b-8c70-736dda5d804a admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:00.297378 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55d6ba-82f4-483b-8c70-736dda5d804a admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:00.327811 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:00.365925 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e415e0f-56e1-4604-9d1f-bf4e40da9f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:00.366531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e415e0f-56e1-4604-9d1f-bf4e40da9f10 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:00.366814 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e415e0f-56e1-4604-9d1f-bf4e40da9f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:00.367089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e415e0f-56e1-4604-9d1f-bf4e40da9f10 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:00.367750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e415e0f-56e1-4604-9d1f-bf4e40da9f10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:00.370274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e415e0f-56e1-4604-9d1f-bf4e40da9f10 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:00.390717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1134/2248] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:00 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:25:00.394818 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68229a-ea59-4026-8437-4613f00c236b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:00.406423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68229a-ea59-4026-8437-4613f00c236b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:00.412761 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76e90a-90e1-4246-bc3a-1e3302b1bd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:00.413461 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76e90a-90e1-4246-bc3a-1e3302b1bd42 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:00.413808 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76e90a-90e1-4246-bc3a-1e3302b1bd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:00.414125 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76e90a-90e1-4246-bc3a-1e3302b1bd42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:00.414870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76e90a-90e1-4246-bc3a-1e3302b1bd42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:00.417316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76e90a-90e1-4246-bc3a-1e3302b1bd42 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:00.427705 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1115/2249] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:00.431710 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e415e0f-56e1-4604-9d1f-bf4e40da9f10 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:00.441823 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e415e0f-56e1-4604-9d1f-bf4e40da9f10 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:00.446533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150474e7-04c1-4261-a487-d6d5200eaec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:00.447122 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150474e7-04c1-4261-a487-d6d5200eaec6 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:00.447406 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150474e7-04c1-4261-a487-d6d5200eaec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:00.447679 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150474e7-04c1-4261-a487-d6d5200eaec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:00.448358 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150474e7-04c1-4261-a487-d6d5200eaec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:00.449969 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150474e7-04c1-4261-a487-d6d5200eaec6 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:00.461501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1135/2250] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:00.466205 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76e90a-90e1-4246-bc3a-1e3302b1bd42 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:00.476245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76e90a-90e1-4246-bc3a-1e3302b1bd42 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:00.481708 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01835cda-b437-47c2-8e5c-6503c2ddee89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:00.482357 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01835cda-b437-47c2-8e5c-6503c2ddee89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:00.482640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01835cda-b437-47c2-8e5c-6503c2ddee89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:00.482917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01835cda-b437-47c2-8e5c-6503c2ddee89 admin admin] PATH_INFO: `/v3/projects/024411b844d6465fb6b1854361591bc6/users/ecadc89eed9443bba41e226708fd6b59/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:00.497716 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01835cda-b437-47c2-8e5c-6503c2ddee89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=024411b844d6465fb6b1854361591bc6, user_id=ecadc89eed9443bba41e226708fd6b59, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:00.500435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01835cda-b437-47c2-8e5c-6503c2ddee89 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:00.513964 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1116/2251] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:00 2022] PUT /identity/v3/projects/024411b844d6465fb6b1854361591bc6/users/ecadc89eed9443bba41e226708fd6b59/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:00.525774 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e24619-0d64-4aaa-a970-28a9d02fa487 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:00.526065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e24619-0d64-4aaa-a970-28a9d02fa487 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:00.526340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e24619-0d64-4aaa-a970-28a9d02fa487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:00.556528 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-19e24619-0d64-4aaa-a970-28a9d02fa487 None None] MFA Rules not processed for user `ecadc89eed9443bba41e226708fd6b59`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:00.557762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-19e24619-0d64-4aaa-a970-28a9d02fa487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:00.625012 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1136/2252] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:00.636626 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01835cda-b437-47c2-8e5c-6503c2ddee89 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:00.638546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-01835cda-b437-47c2-8e5c-6503c2ddee89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:00.698295 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01835cda-b437-47c2-8e5c-6503c2ddee89 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:00.701913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748ee43-7aad-4855-970d-32e6f3d010cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:00.702509 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748ee43-7aad-4855-970d-32e6f3d010cd service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:00.702788 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748ee43-7aad-4855-970d-32e6f3d010cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:00.703062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748ee43-7aad-4855-970d-32e6f3d010cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:00.711527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1748ee43-7aad-4855-970d-32e6f3d010cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:00.713017 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1748ee43-7aad-4855-970d-32e6f3d010cd service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:00.721707 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1117/2253] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:00 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:02.304188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e24619-0d64-4aaa-a970-28a9d02fa487 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:02.305945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-19e24619-0d64-4aaa-a970-28a9d02fa487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:02.371579 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e24619-0d64-4aaa-a970-28a9d02fa487 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:02.376071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094562b-d32f-4b46-a730-c12fa3a4c1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:02.376693 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094562b-d32f-4b46-a730-c12fa3a4c1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:02.376966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094562b-d32f-4b46-a730-c12fa3a4c1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:02.377231 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094562b-d32f-4b46-a730-c12fa3a4c1a2 admin admin] PATH_INFO: `/v3/users/4ab849eaf8a746988ef812a1839f8a42` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:02.378991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b094562b-d32f-4b46-a730-c12fa3a4c1a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ab849eaf8a746988ef812a1839f8a42)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:02.380919 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b094562b-d32f-4b46-a730-c12fa3a4c1a2 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:02.422528 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:02.433157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b094562b-d32f-4b46-a730-c12fa3a4c1a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ab849eaf8a746988ef812a1839f8a42'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:02.433815 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b094562b-d32f-4b46-a730-c12fa3a4c1a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ab849eaf8a746988ef812a1839f8a42'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:02.434423 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b094562b-d32f-4b46-a730-c12fa3a4c1a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ab849eaf8a746988ef812a1839f8a42'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:02.441401 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b094562b-d32f-4b46-a730-c12fa3a4c1a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ab849eaf8a746988ef812a1839f8a42'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:02.453238 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b094562b-d32f-4b46-a730-c12fa3a4c1a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ab849eaf8a746988ef812a1839f8a42'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:02.466781 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1137/2254] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:02 2022] DELETE /identity/v3/users/4ab849eaf8a746988ef812a1839f8a42 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:02.471570 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed08e5-6b12-4360-9462-3bd6172c6f4a None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:02.472125 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed08e5-6b12-4360-9462-3bd6172c6f4a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:02.472229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed08e5-6b12-4360-9462-3bd6172c6f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:02.492331 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e5ed08e5-6b12-4360-9462-3bd6172c6f4a None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:02.494036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e5ed08e5-6b12-4360-9462-3bd6172c6f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:02.552305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1118/2255] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:02.562736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094562b-d32f-4b46-a730-c12fa3a4c1a2 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:02.564154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b094562b-d32f-4b46-a730-c12fa3a4c1a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:02.621919 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094562b-d32f-4b46-a730-c12fa3a4c1a2 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:02.625603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76c252-ce6e-4301-9bb5-f16e62ca8bb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:02.626205 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76c252-ce6e-4301-9bb5-f16e62ca8bb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:02.626485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76c252-ce6e-4301-9bb5-f16e62ca8bb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:02.626760 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76c252-ce6e-4301-9bb5-f16e62ca8bb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:02.635578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76c252-ce6e-4301-9bb5-f16e62ca8bb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:02.637035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76c252-ce6e-4301-9bb5-f16e62ca8bb0 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:02.646428 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1138/2256] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:02.916978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed08e5-6b12-4360-9462-3bd6172c6f4a None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:02.918412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e5ed08e5-6b12-4360-9462-3bd6172c6f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:02.933500 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed08e5-6b12-4360-9462-3bd6172c6f4a None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:02.944305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2dbffc-6075-44aa-9038-ea123117524d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:02.944963 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dbffc-6075-44aa-9038-ea123117524d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:02.945244 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dbffc-6075-44aa-9038-ea123117524d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:02.945518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dbffc-6075-44aa-9038-ea123117524d admin admin] PATH_INFO: `/v3/projects/4702732564084d6faedba47ed90bcb20` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:02.947242 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2dbffc-6075-44aa-9038-ea123117524d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4702732564084d6faedba47ed90bcb20)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:02.949188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2dbffc-6075-44aa-9038-ea123117524d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:02.998774 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7b2dbffc-6075-44aa-9038-ea123117524d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4702732564084d6faedba47ed90bcb20', 'request_id': 'req-7b2dbffc-6075-44aa-9038-ea123117524d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:03.002106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:03.006468 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7b2dbffc-6075-44aa-9038-ea123117524d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4702732564084d6faedba47ed90bcb20', 'request_id': 'req-7b2dbffc-6075-44aa-9038-ea123117524d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:03.014404 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7b2dbffc-6075-44aa-9038-ea123117524d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4702732564084d6faedba47ed90bcb20', 'request_id': 'req-7b2dbffc-6075-44aa-9038-ea123117524d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:03.027095 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7b2dbffc-6075-44aa-9038-ea123117524d admin admin] The token cache is being invalidate because project 4702732564084d6faedba47ed90bcb20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:03.027388 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7b2dbffc-6075-44aa-9038-ea123117524d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:03.028046 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7b2dbffc-6075-44aa-9038-ea123117524d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:03.029538 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1119/2257] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:02 2022] DELETE /identity/v3/projects/4702732564084d6faedba47ed90bcb20 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:25:03.034665 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b01d8-cb0d-4d92-8890-5e84c56a1201 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:03.034998 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b01d8-cb0d-4d92-8890-5e84c56a1201 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:03.035391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b01d8-cb0d-4d92-8890-5e84c56a1201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:03.056565 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-2c1b01d8-cb0d-4d92-8890-5e84c56a1201 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:03.057801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2c1b01d8-cb0d-4d92-8890-5e84c56a1201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:03.118292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1139/2258] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:03.127726 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da0580-c35b-45cc-b2f8-83bf68c86e7f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:03.128048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da0580-c35b-45cc-b2f8-83bf68c86e7f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:03.128350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da0580-c35b-45cc-b2f8-83bf68c86e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:03.150099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-b9da0580-c35b-45cc-b2f8-83bf68c86e7f None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:03.152004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b9da0580-c35b-45cc-b2f8-83bf68c86e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:03.159999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1120/2259] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:03.164669 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b01d8-cb0d-4d92-8890-5e84c56a1201 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:03.174968 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b01d8-cb0d-4d92-8890-5e84c56a1201 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:03.179651 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e8cc0-663d-41e1-a6d6-2734a38c1375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:03.180278 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e8cc0-663d-41e1-a6d6-2734a38c1375 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:03.180675 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e8cc0-663d-41e1-a6d6-2734a38c1375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:03.180955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e8cc0-663d-41e1-a6d6-2734a38c1375 admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:03.181602 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e8cc0-663d-41e1-a6d6-2734a38c1375 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:25:03.181907 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e8cc0-663d-41e1-a6d6-2734a38c1375 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:03.183570 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e8cc0-663d-41e1-a6d6-2734a38c1375 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:03.195091 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1140/2260] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:25:03.199969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da0580-c35b-45cc-b2f8-83bf68c86e7f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:03.211487 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da0580-c35b-45cc-b2f8-83bf68c86e7f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:03.216915 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980b296-39c0-4558-a582-d4b61b74c02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:03.217529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980b296-39c0-4558-a582-d4b61b74c02f admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:03.217809 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980b296-39c0-4558-a582-d4b61b74c02f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:03.218080 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980b296-39c0-4558-a582-d4b61b74c02f admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:03.218718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980b296-39c0-4558-a582-d4b61b74c02f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:03.220668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980b296-39c0-4558-a582-d4b61b74c02f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:03.247135 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1121/2261] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:03 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:03.252058 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e8cc0-663d-41e1-a6d6-2734a38c1375 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:03.263384 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e8cc0-663d-41e1-a6d6-2734a38c1375 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:03.317459 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:03.340178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d905837-0a8d-457b-8adb-4d67b18b1a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:03.341153 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d905837-0a8d-457b-8adb-4d67b18b1a1d admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:03.341529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d905837-0a8d-457b-8adb-4d67b18b1a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:03.341873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d905837-0a8d-457b-8adb-4d67b18b1a1d admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:03.342501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d905837-0a8d-457b-8adb-4d67b18b1a1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:03.344532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d905837-0a8d-457b-8adb-4d67b18b1a1d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:03.376407 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1141/2262] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:03 2022] POST /identity/v3/users => generated 286 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:03.377697 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980b296-39c0-4558-a582-d4b61b74c02f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:03.393324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980b296-39c0-4558-a582-d4b61b74c02f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:03.398021 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497c994-9e2c-4fc4-83c6-4d6a984bf2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:03.398645 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497c994-9e2c-4fc4-83c6-4d6a984bf2bf admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:03.398923 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497c994-9e2c-4fc4-83c6-4d6a984bf2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:03.399230 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497c994-9e2c-4fc4-83c6-4d6a984bf2bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:03.400399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497c994-9e2c-4fc4-83c6-4d6a984bf2bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:03.402004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497c994-9e2c-4fc4-83c6-4d6a984bf2bf admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:03.412204 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1122/2263] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:03.416781 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d905837-0a8d-457b-8adb-4d67b18b1a1d admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:03.426747 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d905837-0a8d-457b-8adb-4d67b18b1a1d admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:03.431537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7fd861-1a6c-477b-8762-7e0a5642b3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:03.432165 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fd861-1a6c-477b-8762-7e0a5642b3cc admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:03.432443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fd861-1a6c-477b-8762-7e0a5642b3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:03.432718 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fd861-1a6c-477b-8762-7e0a5642b3cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:03.433404 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7fd861-1a6c-477b-8762-7e0a5642b3cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:03.434981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7fd861-1a6c-477b-8762-7e0a5642b3cc admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:03.445727 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1142/2264] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:03.449835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497c994-9e2c-4fc4-83c6-4d6a984bf2bf admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:03.462451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497c994-9e2c-4fc4-83c6-4d6a984bf2bf admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:03.467123 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83bb896-7fda-45de-87e1-98dd7aada8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:03.467856 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bb896-7fda-45de-87e1-98dd7aada8f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:03.468137 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bb896-7fda-45de-87e1-98dd7aada8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:03.468412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bb896-7fda-45de-87e1-98dd7aada8f3 admin admin] PATH_INFO: `/v3/projects/97121f2abc1c41ed995d8f5578070495/users/a345f274696f43d6a084d0f1b4fda9d0/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:03.483237 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83bb896-7fda-45de-87e1-98dd7aada8f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97121f2abc1c41ed995d8f5578070495, user_id=a345f274696f43d6a084d0f1b4fda9d0, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:03.485915 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83bb896-7fda-45de-87e1-98dd7aada8f3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:03.499488 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1123/2265] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:03 2022] PUT /identity/v3/projects/97121f2abc1c41ed995d8f5578070495/users/a345f274696f43d6a084d0f1b4fda9d0/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:25:03.510434 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a049d2-80c6-45db-9bbc-80dae055a04b None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:03.510699 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a049d2-80c6-45db-9bbc-80dae055a04b None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:03.510952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a049d2-80c6-45db-9bbc-80dae055a04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:03.538775 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-50a049d2-80c6-45db-9bbc-80dae055a04b None None] MFA Rules not processed for user `a345f274696f43d6a084d0f1b4fda9d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:03.540641 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-50a049d2-80c6-45db-9bbc-80dae055a04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:03.607539 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1143/2266] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:25:03.612133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83bb896-7fda-45de-87e1-98dd7aada8f3 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:03.675639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83bb896-7fda-45de-87e1-98dd7aada8f3 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:03.680387 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da08f82-bce5-4fa4-892f-5e7e16558c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:03.681487 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da08f82-bce5-4fa4-892f-5e7e16558c3c admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:03.681769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da08f82-bce5-4fa4-892f-5e7e16558c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:03.682037 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da08f82-bce5-4fa4-892f-5e7e16558c3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:03.682684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da08f82-bce5-4fa4-892f-5e7e16558c3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:03.684617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da08f82-bce5-4fa4-892f-5e7e16558c3c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:03.714298 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1124/2267] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:03 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:03.718175 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a049d2-80c6-45db-9bbc-80dae055a04b None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:03.728991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a049d2-80c6-45db-9bbc-80dae055a04b None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:03.787650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ede6c1-83b2-41f2-93c8-62362653903c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:03.788218 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ede6c1-83b2-41f2-93c8-62362653903c admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:03.788502 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ede6c1-83b2-41f2-93c8-62362653903c admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:03.789078 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ede6c1-83b2-41f2-93c8-62362653903c admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:03.790695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ede6c1-83b2-41f2-93c8-62362653903c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:03.793131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ede6c1-83b2-41f2-93c8-62362653903c admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:03.814743 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1144/2268] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:03 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:03.818737 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da08f82-bce5-4fa4-892f-5e7e16558c3c admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:03.828917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da08f82-bce5-4fa4-892f-5e7e16558c3c admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:03.833845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c57996-2055-4e74-aea1-921cd9dbd1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:03.834432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c57996-2055-4e74-aea1-921cd9dbd1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:03.834710 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c57996-2055-4e74-aea1-921cd9dbd1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:03.834981 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c57996-2055-4e74-aea1-921cd9dbd1f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:03.835654 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c57996-2055-4e74-aea1-921cd9dbd1f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:03.837285 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c57996-2055-4e74-aea1-921cd9dbd1f1 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:03.847993 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1125/2269] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:03.851787 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ede6c1-83b2-41f2-93c8-62362653903c admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:03.862232 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ede6c1-83b2-41f2-93c8-62362653903c admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:03.867029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076affbe-b4d6-43be-a2e0-023ca6814b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:03.867697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076affbe-b4d6-43be-a2e0-023ca6814b31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:03.868624 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076affbe-b4d6-43be-a2e0-023ca6814b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:03.868918 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076affbe-b4d6-43be-a2e0-023ca6814b31 admin admin] PATH_INFO: `/v3/projects/5038133430d84bd48c6da9ca734737cf/users/b554d8d828c24eeba5dbaae0cd63095e/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:03.884258 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076affbe-b4d6-43be-a2e0-023ca6814b31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5038133430d84bd48c6da9ca734737cf, user_id=b554d8d828c24eeba5dbaae0cd63095e, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:03.886941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076affbe-b4d6-43be-a2e0-023ca6814b31 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:03.900537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1145/2270] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:03 2022] PUT /identity/v3/projects/5038133430d84bd48c6da9ca734737cf/users/b554d8d828c24eeba5dbaae0cd63095e/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:03.904457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c57996-2055-4e74-aea1-921cd9dbd1f1 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:03.914676 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c57996-2055-4e74-aea1-921cd9dbd1f1 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:03.963064 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:03.974128 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff2b4a-269a-4d65-a2b9-a936bfb7f01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:03.974749 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff2b4a-269a-4d65-a2b9-a936bfb7f01c admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:03.975016 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff2b4a-269a-4d65-a2b9-a936bfb7f01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:03.975288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff2b4a-269a-4d65-a2b9-a936bfb7f01c admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:03.975965 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff2b4a-269a-4d65-a2b9-a936bfb7f01c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:03.977598 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff2b4a-269a-4d65-a2b9-a936bfb7f01c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:03.987279 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1126/2271] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:03.991551 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076affbe-b4d6-43be-a2e0-023ca6814b31 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:04.002391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076affbe-b4d6-43be-a2e0-023ca6814b31 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:04.007003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912f5b0-1518-4217-93d8-00bd6484b84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:04.007647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912f5b0-1518-4217-93d8-00bd6484b84a admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:04.007951 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912f5b0-1518-4217-93d8-00bd6484b84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:04.008221 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912f5b0-1518-4217-93d8-00bd6484b84a admin admin] PATH_INFO: `/v3/projects/5038133430d84bd48c6da9ca734737cf/users/b554d8d828c24eeba5dbaae0cd63095e/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:04.010639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7912f5b0-1518-4217-93d8-00bd6484b84a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5038133430d84bd48c6da9ca734737cf, user_id=b554d8d828c24eeba5dbaae0cd63095e, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:04.013259 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7912f5b0-1518-4217-93d8-00bd6484b84a admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:04.023228 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1146/2272] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:03 2022] PUT /identity/v3/projects/5038133430d84bd48c6da9ca734737cf/users/b554d8d828c24eeba5dbaae0cd63095e/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:04.035230 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303d7b9-39e6-4e27-806c-27f0f75da4e1 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:04.035524 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303d7b9-39e6-4e27-806c-27f0f75da4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:04.035832 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303d7b9-39e6-4e27-806c-27f0f75da4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:04.064620 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-5303d7b9-39e6-4e27-806c-27f0f75da4e1 None None] MFA Rules not processed for user `b554d8d828c24eeba5dbaae0cd63095e`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:04.065830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5303d7b9-39e6-4e27-806c-27f0f75da4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:04.141718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1127/2273] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:04.153979 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912f5b0-1518-4217-93d8-00bd6484b84a admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:04.155919 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-7912f5b0-1518-4217-93d8-00bd6484b84a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:04.162543 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:04.216695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912f5b0-1518-4217-93d8-00bd6484b84a admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:04.220932 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714b15a-7c24-4a59-848d-c5c1ede0695a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:04.221556 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714b15a-7c24-4a59-848d-c5c1ede0695a service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:04.221830 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714b15a-7c24-4a59-848d-c5c1ede0695a service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:04.222100 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714b15a-7c24-4a59-848d-c5c1ede0695a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:04.232266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714b15a-7c24-4a59-848d-c5c1ede0695a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:04.233688 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714b15a-7c24-4a59-848d-c5c1ede0695a service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:04.242864 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1147/2274] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:04.319373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303d7b9-39e6-4e27-806c-27f0f75da4e1 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:04.329163 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303d7b9-39e6-4e27-806c-27f0f75da4e1 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:04.332730 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2186f-cd50-491a-9bc5-62edcd641c47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:04.333617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2186f-cd50-491a-9bc5-62edcd641c47 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:04.333895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2186f-cd50-491a-9bc5-62edcd641c47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:04.334266 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2186f-cd50-491a-9bc5-62edcd641c47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:04.393741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2186f-cd50-491a-9bc5-62edcd641c47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:04.395435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2186f-cd50-491a-9bc5-62edcd641c47 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:04.406349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1128/2275] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:14.238746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=24 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:14.401552 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=68 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:22.144656 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714b15a-7c24-4a59-848d-c5c1ede0695a service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:22.146084 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a714b15a-7c24-4a59-848d-c5c1ede0695a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:22.214937 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714b15a-7c24-4a59-848d-c5c1ede0695a service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:22.219876 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ad925-c5f8-4760-a254-f89e077ac018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:22.220504 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ad925-c5f8-4760-a254-f89e077ac018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:22.220778 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ad925-c5f8-4760-a254-f89e077ac018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:22.221048 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ad925-c5f8-4760-a254-f89e077ac018 admin admin] PATH_INFO: `/v3/users/ecadc89eed9443bba41e226708fd6b59` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:22.222224 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ad925-c5f8-4760-a254-f89e077ac018 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecadc89eed9443bba41e226708fd6b59)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:22.224148 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ad925-c5f8-4760-a254-f89e077ac018 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:22.271950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-286ad925-c5f8-4760-a254-f89e077ac018 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecadc89eed9443bba41e226708fd6b59'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:22.272610 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-286ad925-c5f8-4760-a254-f89e077ac018 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecadc89eed9443bba41e226708fd6b59'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:22.273232 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-286ad925-c5f8-4760-a254-f89e077ac018 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecadc89eed9443bba41e226708fd6b59'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:22.281209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-286ad925-c5f8-4760-a254-f89e077ac018 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecadc89eed9443bba41e226708fd6b59'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:22.294390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-286ad925-c5f8-4760-a254-f89e077ac018 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecadc89eed9443bba41e226708fd6b59'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:22.307409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1148/2276] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:22 2022] DELETE /identity/v3/users/ecadc89eed9443bba41e226708fd6b59 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:22.312412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c2cd0-6120-4110-8fdb-14cd000ade3e None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:22.312869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c2cd0-6120-4110-8fdb-14cd000ade3e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:22.313134 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c2cd0-6120-4110-8fdb-14cd000ade3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:22.335585 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-fc9c2cd0-6120-4110-8fdb-14cd000ade3e None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:22.337364 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-fc9c2cd0-6120-4110-8fdb-14cd000ade3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:22.397619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1129/2277] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:22.407241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ad925-c5f8-4760-a254-f89e077ac018 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:22.408659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-286ad925-c5f8-4760-a254-f89e077ac018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:22.472142 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ad925-c5f8-4760-a254-f89e077ac018 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:22.475943 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a322baf-b6b8-4ffb-a1e6-5d2847542c17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:22.477168 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a322baf-b6b8-4ffb-a1e6-5d2847542c17 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:22.477501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a322baf-b6b8-4ffb-a1e6-5d2847542c17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:22.477840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a322baf-b6b8-4ffb-a1e6-5d2847542c17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:22.489637 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a322baf-b6b8-4ffb-a1e6-5d2847542c17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:22.489637 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a322baf-b6b8-4ffb-a1e6-5d2847542c17 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:22.500552 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1149/2278] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:22.852883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c2cd0-6120-4110-8fdb-14cd000ade3e None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:22.854514 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-fc9c2cd0-6120-4110-8fdb-14cd000ade3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:22.881137 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c2cd0-6120-4110-8fdb-14cd000ade3e None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:22.889444 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a2f24-87eb-4791-9272-c03ff737f347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:22.890514 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a2f24-87eb-4791-9272-c03ff737f347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:22.890514 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a2f24-87eb-4791-9272-c03ff737f347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:22.890710 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a2f24-87eb-4791-9272-c03ff737f347 admin admin] PATH_INFO: `/v3/projects/024411b844d6465fb6b1854361591bc6` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:22.892030 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a2f24-87eb-4791-9272-c03ff737f347 admin admin] RBAC: Authorizing `identity:delete_project(project_id=024411b844d6465fb6b1854361591bc6)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:22.894300 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a2f24-87eb-4791-9272-c03ff737f347 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:22.959017 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8c7a2f24-87eb-4791-9272-c03ff737f347 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '024411b844d6465fb6b1854361591bc6', 'request_id': 'req-8c7a2f24-87eb-4791-9272-c03ff737f347'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:22.968960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8c7a2f24-87eb-4791-9272-c03ff737f347 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '024411b844d6465fb6b1854361591bc6', 'request_id': 'req-8c7a2f24-87eb-4791-9272-c03ff737f347'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:22.984691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8c7a2f24-87eb-4791-9272-c03ff737f347 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '024411b844d6465fb6b1854361591bc6', 'request_id': 'req-8c7a2f24-87eb-4791-9272-c03ff737f347'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:22.998799 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8c7a2f24-87eb-4791-9272-c03ff737f347 admin admin] The token cache is being invalidate because project 024411b844d6465fb6b1854361591bc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:22.999183 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8c7a2f24-87eb-4791-9272-c03ff737f347 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:23.000305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8c7a2f24-87eb-4791-9272-c03ff737f347 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:23.002513 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1130/2279] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:22 2022] DELETE /identity/v3/projects/024411b844d6465fb6b1854361591bc6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:25.424958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391b12e-4d54-4a78-aab2-753ad20b53e0 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:25.425452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391b12e-4d54-4a78-aab2-753ad20b53e0 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:25.425538 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391b12e-4d54-4a78-aab2-753ad20b53e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:25.445129 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-0391b12e-4d54-4a78-aab2-753ad20b53e0 None None] MFA Rules not processed for user `9f160368e1984319ad4fa37c2794a956`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:25.446891 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-0391b12e-4d54-4a78-aab2-753ad20b53e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:25.463059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:25.504209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1150/2280] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:25:32.560690 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a2f24-87eb-4791-9272-c03ff737f347 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:32.562241 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8c7a2f24-87eb-4791-9272-c03ff737f347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:32.630919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a2f24-87eb-4791-9272-c03ff737f347 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:32.637267 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc22b39-867e-442a-a9b0-1b39f6162127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:32.637920 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc22b39-867e-442a-a9b0-1b39f6162127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:32.638433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc22b39-867e-442a-a9b0-1b39f6162127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:32.638716 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc22b39-867e-442a-a9b0-1b39f6162127 admin admin] PATH_INFO: `/v3/users/425ad4f89595428e816ce005de4c6241` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:32.639900 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc22b39-867e-442a-a9b0-1b39f6162127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=425ad4f89595428e816ce005de4c6241)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:32.642637 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc22b39-867e-442a-a9b0-1b39f6162127 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:32.678299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:32.700882 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cfc22b39-867e-442a-a9b0-1b39f6162127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '425ad4f89595428e816ce005de4c6241'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:32.709570 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cfc22b39-867e-442a-a9b0-1b39f6162127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '425ad4f89595428e816ce005de4c6241'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:32.710266 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cfc22b39-867e-442a-a9b0-1b39f6162127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '425ad4f89595428e816ce005de4c6241'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:32.711428 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cfc22b39-867e-442a-a9b0-1b39f6162127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '425ad4f89595428e816ce005de4c6241'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:32.728882 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-cfc22b39-867e-442a-a9b0-1b39f6162127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '425ad4f89595428e816ce005de4c6241'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:32.746115 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1131/2281] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:32 2022] DELETE /identity/v3/users/425ad4f89595428e816ce005de4c6241 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:32.750922 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a806d-a99f-4488-8609-b8a8b7268a98 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:32.751865 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a806d-a99f-4488-8609-b8a8b7268a98 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:32.752367 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a806d-a99f-4488-8609-b8a8b7268a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:32.774639 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-066a806d-a99f-4488-8609-b8a8b7268a98 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:32.776366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-066a806d-a99f-4488-8609-b8a8b7268a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:32.844568 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1151/2282] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:32.854505 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc22b39-867e-442a-a9b0-1b39f6162127 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:32.856368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cfc22b39-867e-442a-a9b0-1b39f6162127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:32.926171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc22b39-867e-442a-a9b0-1b39f6162127 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:32.931173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241c5eb-4fea-4ef7-bc2c-5f47dcda7a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:32.931895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241c5eb-4fea-4ef7-bc2c-5f47dcda7a76 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:32.932247 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241c5eb-4fea-4ef7-bc2c-5f47dcda7a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:32.932589 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241c5eb-4fea-4ef7-bc2c-5f47dcda7a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:32.944454 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241c5eb-4fea-4ef7-bc2c-5f47dcda7a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:32.947393 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241c5eb-4fea-4ef7-bc2c-5f47dcda7a76 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:32.963400 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1132/2283] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:33.259390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a806d-a99f-4488-8609-b8a8b7268a98 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:33.261059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-066a806d-a99f-4488-8609-b8a8b7268a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:33.273725 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a806d-a99f-4488-8609-b8a8b7268a98 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:33.278329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf3ceb-f178-4432-9e19-76cd21141a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:33.278949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf3ceb-f178-4432-9e19-76cd21141a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:33.279229 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf3ceb-f178-4432-9e19-76cd21141a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:33.279499 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf3ceb-f178-4432-9e19-76cd21141a54 admin admin] PATH_INFO: `/v3/projects/0efa2de623ef4eea8055263ced591b55` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:33.280673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf3ceb-f178-4432-9e19-76cd21141a54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0efa2de623ef4eea8055263ced591b55)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:33.282617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf3ceb-f178-4432-9e19-76cd21141a54 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:33.329373 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-50cf3ceb-f178-4432-9e19-76cd21141a54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0efa2de623ef4eea8055263ced591b55', 'request_id': 'req-50cf3ceb-f178-4432-9e19-76cd21141a54'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:33.336032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-50cf3ceb-f178-4432-9e19-76cd21141a54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0efa2de623ef4eea8055263ced591b55', 'request_id': 'req-50cf3ceb-f178-4432-9e19-76cd21141a54'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:33.343480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-50cf3ceb-f178-4432-9e19-76cd21141a54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0efa2de623ef4eea8055263ced591b55', 'request_id': 'req-50cf3ceb-f178-4432-9e19-76cd21141a54'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:33.357900 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-50cf3ceb-f178-4432-9e19-76cd21141a54 admin admin] The token cache is being invalidate because project 0efa2de623ef4eea8055263ced591b55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:33.358267 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-50cf3ceb-f178-4432-9e19-76cd21141a54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:33.358952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-50cf3ceb-f178-4432-9e19-76cd21141a54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:33.361416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1152/2284] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:33 2022] DELETE /identity/v3/projects/0efa2de623ef4eea8055263ced591b55 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:33.368268 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30528753-5e1f-4f4f-be70-7e37a774cd14 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:33.368556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30528753-5e1f-4f4f-be70-7e37a774cd14 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:33.368827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30528753-5e1f-4f4f-be70-7e37a774cd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:33.388087 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-30528753-5e1f-4f4f-be70-7e37a774cd14 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:33.389306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-30528753-5e1f-4f4f-be70-7e37a774cd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:33.448351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1133/2285] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:25:33.456945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599d28f-b697-4005-bedc-f5a0f7dacc55 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:33.457233 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599d28f-b697-4005-bedc-f5a0f7dacc55 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:33.457506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599d28f-b697-4005-bedc-f5a0f7dacc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:33.479913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-f599d28f-b697-4005-bedc-f5a0f7dacc55 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:33.481140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f599d28f-b697-4005-bedc-f5a0f7dacc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:33.489302 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1153/2286] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:33.495689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30528753-5e1f-4f4f-be70-7e37a774cd14 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:33.506082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30528753-5e1f-4f4f-be70-7e37a774cd14 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:33.523786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea86fd-b49a-46c8-9d71-ebaf3fc81a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:33.523786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea86fd-b49a-46c8-9d71-ebaf3fc81a94 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:33.523786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea86fd-b49a-46c8-9d71-ebaf3fc81a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:33.523786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea86fd-b49a-46c8-9d71-ebaf3fc81a94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:33.524213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea86fd-b49a-46c8-9d71-ebaf3fc81a94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:25:33.524213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea86fd-b49a-46c8-9d71-ebaf3fc81a94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:33.524213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea86fd-b49a-46c8-9d71-ebaf3fc81a94 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:33.529633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1134/2287] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:25:33.533676 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599d28f-b697-4005-bedc-f5a0f7dacc55 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:33.543902 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599d28f-b697-4005-bedc-f5a0f7dacc55 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:33.548705 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a367f5-d3ec-4067-85a3-68d92d4702f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:33.549328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a367f5-d3ec-4067-85a3-68d92d4702f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:33.549614 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a367f5-d3ec-4067-85a3-68d92d4702f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:33.549884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a367f5-d3ec-4067-85a3-68d92d4702f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:33.550524 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a367f5-d3ec-4067-85a3-68d92d4702f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:33.552474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a367f5-d3ec-4067-85a3-68d92d4702f8 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:33.580035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1154/2288] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:33 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:33.584500 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea86fd-b49a-46c8-9d71-ebaf3fc81a94 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:33.595611 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea86fd-b49a-46c8-9d71-ebaf3fc81a94 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:33.625667 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:33.653464 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da085c3-35d8-4906-b415-7cb75aaa7869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:33.654070 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da085c3-35d8-4906-b415-7cb75aaa7869 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:33.654352 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da085c3-35d8-4906-b415-7cb75aaa7869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:33.654622 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da085c3-35d8-4906-b415-7cb75aaa7869 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:33.655283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da085c3-35d8-4906-b415-7cb75aaa7869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:33.657227 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da085c3-35d8-4906-b415-7cb75aaa7869 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:33.682423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1135/2289] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:33 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:33.684701 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a367f5-d3ec-4067-85a3-68d92d4702f8 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:33.695475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a367f5-d3ec-4067-85a3-68d92d4702f8 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:33.700259 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b778f-9d0e-42d5-a9ea-7a0f13bfc0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:33.700871 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b778f-9d0e-42d5-a9ea-7a0f13bfc0b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:33.701146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b778f-9d0e-42d5-a9ea-7a0f13bfc0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:33.701441 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b778f-9d0e-42d5-a9ea-7a0f13bfc0b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:33.702072 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b778f-9d0e-42d5-a9ea-7a0f13bfc0b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:33.703637 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b778f-9d0e-42d5-a9ea-7a0f13bfc0b9 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:33.714298 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1155/2290] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:33.718186 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da085c3-35d8-4906-b415-7cb75aaa7869 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:33.728738 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da085c3-35d8-4906-b415-7cb75aaa7869 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:33.733510 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc038c22-3507-4c94-a642-15a323b2f5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:33.734906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc038c22-3507-4c94-a642-15a323b2f5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:33.735192 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc038c22-3507-4c94-a642-15a323b2f5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:33.735463 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc038c22-3507-4c94-a642-15a323b2f5f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:33.736171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc038c22-3507-4c94-a642-15a323b2f5f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:33.737826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc038c22-3507-4c94-a642-15a323b2f5f3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:33.749147 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1136/2291] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:33.753289 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b778f-9d0e-42d5-a9ea-7a0f13bfc0b9 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:33.763581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b778f-9d0e-42d5-a9ea-7a0f13bfc0b9 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:33.768170 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb432abf-1892-456c-9c5b-2175ba191504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:33.768806 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb432abf-1892-456c-9c5b-2175ba191504 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:33.769084 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb432abf-1892-456c-9c5b-2175ba191504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:33.769360 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb432abf-1892-456c-9c5b-2175ba191504 admin admin] PATH_INFO: `/v3/projects/221fd300d74c4248a1bfc9b785476477/users/6e8793d50fce45f08c0c4000b7ae75e4/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:33.778240 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:33.785794 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb432abf-1892-456c-9c5b-2175ba191504 admin admin] RBAC: Authorizing `identity:create_grant(project_id=221fd300d74c4248a1bfc9b785476477, user_id=6e8793d50fce45f08c0c4000b7ae75e4, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:33.789006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb432abf-1892-456c-9c5b-2175ba191504 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:33.801715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1156/2292] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:33 2022] PUT /identity/v3/projects/221fd300d74c4248a1bfc9b785476477/users/6e8793d50fce45f08c0c4000b7ae75e4/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:25:33.806851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f76f2b-5f20-4163-b098-25576ded5777 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:33.807133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f76f2b-5f20-4163-b098-25576ded5777 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:33.807406 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f76f2b-5f20-4163-b098-25576ded5777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:33.826254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c0f76f2b-5f20-4163-b098-25576ded5777 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:33.827447 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c0f76f2b-5f20-4163-b098-25576ded5777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:33.884969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1137/2293] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:33.894483 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb432abf-1892-456c-9c5b-2175ba191504 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:33.896216 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-fb432abf-1892-456c-9c5b-2175ba191504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:33.952267 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb432abf-1892-456c-9c5b-2175ba191504 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:33.955959 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b06b74-0898-4a3a-bdc0-358036d414c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:33.956555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b06b74-0898-4a3a-bdc0-358036d414c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:33.956837 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b06b74-0898-4a3a-bdc0-358036d414c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:33.957109 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b06b74-0898-4a3a-bdc0-358036d414c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:33.966803 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b06b74-0898-4a3a-bdc0-358036d414c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:33.968260 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b06b74-0898-4a3a-bdc0-358036d414c0 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:33.977661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1157/2294] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:38.140574 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4b6ff-6f2f-48a6-8911-4d2f0f4a6ceb None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:38.141082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4b6ff-6f2f-48a6-8911-4d2f0f4a6ceb None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:38.141193 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4b6ff-6f2f-48a6-8911-4d2f0f4a6ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:38.173235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e1f4b6ff-6f2f-48a6-8911-4d2f0f4a6ceb None None] MFA Rules not processed for user `6e8793d50fce45f08c0c4000b7ae75e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:38.174477 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e1f4b6ff-6f2f-48a6-8911-4d2f0f4a6ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:38.245447 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1138/2295] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:38.250090 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b06b74-0898-4a3a-bdc0-358036d414c0 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:38.261518 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b06b74-0898-4a3a-bdc0-358036d414c0 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:38.266510 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf3681-c95b-431c-9eb9-3886c63c2558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:38.267223 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf3681-c95b-431c-9eb9-3886c63c2558 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:38.267564 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf3681-c95b-431c-9eb9-3886c63c2558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:38.267981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf3681-c95b-431c-9eb9-3886c63c2558 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:38.268840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf3681-c95b-431c-9eb9-3886c63c2558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:38.271106 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf3681-c95b-431c-9eb9-3886c63c2558 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:38.303175 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1158/2296] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:38 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:38.305003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4b6ff-6f2f-48a6-8911-4d2f0f4a6ceb None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:38.316100 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4b6ff-6f2f-48a6-8911-4d2f0f4a6ceb None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:38.382478 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:38.406984 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8f6ad-9706-474b-8aeb-9dedaf595a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:38.407689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8f6ad-9706-474b-8aeb-9dedaf595a3b admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:38.408070 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8f6ad-9706-474b-8aeb-9dedaf595a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:38.408489 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8f6ad-9706-474b-8aeb-9dedaf595a3b admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:38.409235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8f6ad-9706-474b-8aeb-9dedaf595a3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:38.412232 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8f6ad-9706-474b-8aeb-9dedaf595a3b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:38.439919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1139/2297] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:38 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:38.444283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf3681-c95b-431c-9eb9-3886c63c2558 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:38.458034 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf3681-c95b-431c-9eb9-3886c63c2558 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:38.464596 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19a8e3-3912-462d-90bf-55476a46b63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:38.465209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19a8e3-3912-462d-90bf-55476a46b63a admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:38.466082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19a8e3-3912-462d-90bf-55476a46b63a admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:38.466341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19a8e3-3912-462d-90bf-55476a46b63a admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:38.467002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f19a8e3-3912-462d-90bf-55476a46b63a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:38.469404 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f19a8e3-3912-462d-90bf-55476a46b63a admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:38.482286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1159/2298] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:38.486205 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8f6ad-9706-474b-8aeb-9dedaf595a3b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:38.497602 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8f6ad-9706-474b-8aeb-9dedaf595a3b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:38.502438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649c2a0-1e05-4b72-a501-d99dc9affcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:38.503111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649c2a0-1e05-4b72-a501-d99dc9affcf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:38.503393 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649c2a0-1e05-4b72-a501-d99dc9affcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:38.503666 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649c2a0-1e05-4b72-a501-d99dc9affcf9 admin admin] PATH_INFO: `/v3/projects/8b760388f536433ba529aa20b48b8c63/users/c0d9f7bb5d334a3ab57f1ec0e1bc6276/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:38.526350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649c2a0-1e05-4b72-a501-d99dc9affcf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b760388f536433ba529aa20b48b8c63, user_id=c0d9f7bb5d334a3ab57f1ec0e1bc6276, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:38.529178 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649c2a0-1e05-4b72-a501-d99dc9affcf9 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:38.546305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1140/2299] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:38 2022] PUT /identity/v3/projects/8b760388f536433ba529aa20b48b8c63/users/c0d9f7bb5d334a3ab57f1ec0e1bc6276/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:38.550134 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19a8e3-3912-462d-90bf-55476a46b63a admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:38.562740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19a8e3-3912-462d-90bf-55476a46b63a admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:38.621195 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bea40-2d93-4f10-a5c8-b3bfb1b6b164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:38.621727 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bea40-2d93-4f10-a5c8-b3bfb1b6b164 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:38.622009 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bea40-2d93-4f10-a5c8-b3bfb1b6b164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:38.622285 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bea40-2d93-4f10-a5c8-b3bfb1b6b164 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:38.622952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bea40-2d93-4f10-a5c8-b3bfb1b6b164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:38.624723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bea40-2d93-4f10-a5c8-b3bfb1b6b164 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:38.639718 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1160/2300] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:38.643784 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649c2a0-1e05-4b72-a501-d99dc9affcf9 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:38.654958 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649c2a0-1e05-4b72-a501-d99dc9affcf9 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:38.659802 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df4090-f666-422a-97db-7757f4d62f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:38.660469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df4090-f666-422a-97db-7757f4d62f6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:38.660749 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df4090-f666-422a-97db-7757f4d62f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:38.661024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df4090-f666-422a-97db-7757f4d62f6b admin admin] PATH_INFO: `/v3/projects/8b760388f536433ba529aa20b48b8c63/users/c0d9f7bb5d334a3ab57f1ec0e1bc6276/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:38.662978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df4090-f666-422a-97db-7757f4d62f6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b760388f536433ba529aa20b48b8c63, user_id=c0d9f7bb5d334a3ab57f1ec0e1bc6276, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:38.666269 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df4090-f666-422a-97db-7757f4d62f6b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:38.677911 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1141/2301] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:38 2022] PUT /identity/v3/projects/8b760388f536433ba529aa20b48b8c63/users/c0d9f7bb5d334a3ab57f1ec0e1bc6276/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:44.016929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a6092-ff73-47e2-86c7-73d051176e2b None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:44.018100 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a6092-ff73-47e2-86c7-73d051176e2b None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:44.018389 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a6092-ff73-47e2-86c7-73d051176e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:44.048591 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-b90a6092-ff73-47e2-86c7-73d051176e2b None None] MFA Rules not processed for user `c0d9f7bb5d334a3ab57f1ec0e1bc6276`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:44.049838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b90a6092-ff73-47e2-86c7-73d051176e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:44.101585 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:44.135122 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1161/2302] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:44.149731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df4090-f666-422a-97db-7757f4d62f6b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:44.151938 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-41df4090-f666-422a-97db-7757f4d62f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:44.249809 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df4090-f666-422a-97db-7757f4d62f6b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:44.256486 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0ebb7-9948-4864-a447-aabc1a01e8d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:44.258255 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0ebb7-9948-4864-a447-aabc1a01e8d6 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:44.258255 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0ebb7-9948-4864-a447-aabc1a01e8d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:44.258482 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0ebb7-9948-4864-a447-aabc1a01e8d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:44.335092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0ebb7-9948-4864-a447-aabc1a01e8d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:44.337321 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0ebb7-9948-4864-a447-aabc1a01e8d6 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:44.349563 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1142/2303] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:45.960941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f9674-f7d5-4992-9908-15a948103c5d None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:45.961519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f9674-f7d5-4992-9908-15a948103c5d None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:45.961519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f9674-f7d5-4992-9908-15a948103c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:45.983962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-4f9f9674-f7d5-4992-9908-15a948103c5d None None] MFA Rules not processed for user `6e8793d50fce45f08c0c4000b7ae75e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:45.985669 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-4f9f9674-f7d5-4992-9908-15a948103c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:46.002711 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1162/2304] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:54.095586 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0ebb7-9948-4864-a447-aabc1a01e8d6 service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:54.097143 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b8e0ebb7-9948-4864-a447-aabc1a01e8d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:54.129878 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d095787-9387-4823-885d-6bf18f220084 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:54.130378 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d095787-9387-4823-885d-6bf18f220084 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:54.130485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d095787-9387-4823-885d-6bf18f220084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:54.134908 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:54.151021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-1d095787-9387-4823-885d-6bf18f220084 None None] MFA Rules not processed for user `06d279b488d148428b7a3c35a9786ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:54.152393 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-1d095787-9387-4823-885d-6bf18f220084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:54.175649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0ebb7-9948-4864-a447-aabc1a01e8d6 service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:54.180993 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59fb1b-ef1c-491e-84b2-609ccf9b8269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:54.181643 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59fb1b-ef1c-491e-84b2-609ccf9b8269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:54.181924 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59fb1b-ef1c-491e-84b2-609ccf9b8269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:54.182201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59fb1b-ef1c-491e-84b2-609ccf9b8269 admin admin] PATH_INFO: `/v3/users/a345f274696f43d6a084d0f1b4fda9d0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:54.183335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59fb1b-ef1c-491e-84b2-609ccf9b8269 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a345f274696f43d6a084d0f1b4fda9d0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:54.185572 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59fb1b-ef1c-491e-84b2-609ccf9b8269 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:54.224169 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1163/2305] 10.222.0.81 () {58 vars in 1009 bytes} [Mon Sep 19 11:25:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:25:54.234813 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7b59fb1b-ef1c-491e-84b2-609ccf9b8269 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a345f274696f43d6a084d0f1b4fda9d0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:54.243952 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7b59fb1b-ef1c-491e-84b2-609ccf9b8269 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a345f274696f43d6a084d0f1b4fda9d0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:54.244627 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7b59fb1b-ef1c-491e-84b2-609ccf9b8269 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a345f274696f43d6a084d0f1b4fda9d0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:54.245319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7b59fb1b-ef1c-491e-84b2-609ccf9b8269 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a345f274696f43d6a084d0f1b4fda9d0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:54.263443 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7b59fb1b-ef1c-491e-84b2-609ccf9b8269 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a345f274696f43d6a084d0f1b4fda9d0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:54.271614 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d095787-9387-4823-885d-6bf18f220084 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:54.273426 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-1d095787-9387-4823-885d-6bf18f220084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:54.276515 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1143/2306] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:54 2022] DELETE /identity/v3/users/a345f274696f43d6a084d0f1b4fda9d0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:54.281050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39d382-4a74-42a0-a769-c33a44566e22 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:54.281352 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39d382-4a74-42a0-a769-c33a44566e22 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:54.281623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39d382-4a74-42a0-a769-c33a44566e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:54.303125 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-4d39d382-4a74-42a0-a769-c33a44566e22 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:54.304402 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4d39d382-4a74-42a0-a769-c33a44566e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:54.356227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d095787-9387-4823-885d-6bf18f220084 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:54.369856 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1144/2307] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:25:54.379639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39d382-4a74-42a0-a769-c33a44566e22 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:54.380993 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4d39d382-4a74-42a0-a769-c33a44566e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:54.420991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5455f9b-ac82-48bd-8873-6f4951c44f3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8a9732807d4921a41a6a2d5019e2f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:54.421610 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5455f9b-ac82-48bd-8873-6f4951c44f3d service swift] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:54.421890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5455f9b-ac82-48bd-8873-6f4951c44f3d service swift] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:54.422162 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5455f9b-ac82-48bd-8873-6f4951c44f3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:54.425121 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c5455f9b-ac82-48bd-8873-6f4951c44f3d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:54.439268 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39d382-4a74-42a0-a769-c33a44566e22 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:54.443013 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755cce3-db06-4152-8b9f-b332fa778139 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:54.443612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755cce3-db06-4152-8b9f-b332fa778139 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:54.443983 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755cce3-db06-4152-8b9f-b332fa778139 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:54.444254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755cce3-db06-4152-8b9f-b332fa778139 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:54.448533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:54.450283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:54.454302 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755cce3-db06-4152-8b9f-b332fa778139 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:54.457276 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755cce3-db06-4152-8b9f-b332fa778139 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:54.469745 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1145/2308] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:25:54.493571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5455f9b-ac82-48bd-8873-6f4951c44f3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:54.498829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5455f9b-ac82-48bd-8873-6f4951c44f3d service swift] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:54.507939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1164/2309] 10.222.0.81 () {60 vars in 1384 bytes} [Mon Sep 19 11:25:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 11:25:55.096300 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755cce3-db06-4152-8b9f-b332fa778139 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:55.097751 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c755cce3-db06-4152-8b9f-b332fa778139 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:55.129544 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755cce3-db06-4152-8b9f-b332fa778139 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:55.139533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842c657-4ee4-4330-b5f6-c58eb893715a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:55.140246 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842c657-4ee4-4330-b5f6-c58eb893715a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:55.140529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842c657-4ee4-4330-b5f6-c58eb893715a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:55.140808 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842c657-4ee4-4330-b5f6-c58eb893715a admin admin] PATH_INFO: `/v3/projects/97121f2abc1c41ed995d8f5578070495` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:55.141941 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c842c657-4ee4-4330-b5f6-c58eb893715a admin admin] RBAC: Authorizing `identity:delete_project(project_id=97121f2abc1c41ed995d8f5578070495)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:55.143918 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c842c657-4ee4-4330-b5f6-c58eb893715a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:55.203791 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c842c657-4ee4-4330-b5f6-c58eb893715a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97121f2abc1c41ed995d8f5578070495', 'request_id': 'req-c842c657-4ee4-4330-b5f6-c58eb893715a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:55.211230 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c842c657-4ee4-4330-b5f6-c58eb893715a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97121f2abc1c41ed995d8f5578070495', 'request_id': 'req-c842c657-4ee4-4330-b5f6-c58eb893715a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:55.218670 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c842c657-4ee4-4330-b5f6-c58eb893715a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97121f2abc1c41ed995d8f5578070495', 'request_id': 'req-c842c657-4ee4-4330-b5f6-c58eb893715a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:55.236609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c842c657-4ee4-4330-b5f6-c58eb893715a admin admin] The token cache is being invalidate because project 97121f2abc1c41ed995d8f5578070495 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:55.237265 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c842c657-4ee4-4330-b5f6-c58eb893715a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:55.238122 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c842c657-4ee4-4330-b5f6-c58eb893715a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:55.240158 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1146/2310] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:55 2022] DELETE /identity/v3/projects/97121f2abc1c41ed995d8f5578070495 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:55.244985 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5455f9b-ac82-48bd-8873-6f4951c44f3d service swift] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:55.256910 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5455f9b-ac82-48bd-8873-6f4951c44f3d service swift] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:55.347369 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179b66a-4375-4c6b-82b1-7a368cf28178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:55.347945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179b66a-4375-4c6b-82b1-7a368cf28178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:55.348234 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179b66a-4375-4c6b-82b1-7a368cf28178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:55.348511 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179b66a-4375-4c6b-82b1-7a368cf28178 admin admin] PATH_INFO: `/v3/users/b554d8d828c24eeba5dbaae0cd63095e` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:55.349665 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179b66a-4375-4c6b-82b1-7a368cf28178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b554d8d828c24eeba5dbaae0cd63095e)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:55.351671 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179b66a-4375-4c6b-82b1-7a368cf28178 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:55.418744 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-f179b66a-4375-4c6b-82b1-7a368cf28178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b554d8d828c24eeba5dbaae0cd63095e'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:55.419415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-f179b66a-4375-4c6b-82b1-7a368cf28178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b554d8d828c24eeba5dbaae0cd63095e'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:55.420097 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-f179b66a-4375-4c6b-82b1-7a368cf28178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b554d8d828c24eeba5dbaae0cd63095e'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:55.429190 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-f179b66a-4375-4c6b-82b1-7a368cf28178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b554d8d828c24eeba5dbaae0cd63095e'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:55.446403 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-f179b66a-4375-4c6b-82b1-7a368cf28178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b554d8d828c24eeba5dbaae0cd63095e'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:55.472142 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1165/2311] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:25:55 2022] DELETE /identity/v3/users/b554d8d828c24eeba5dbaae0cd63095e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:55.823998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842c657-4ee4-4330-b5f6-c58eb893715a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:55.827678 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c842c657-4ee4-4330-b5f6-c58eb893715a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:55.862953 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842c657-4ee4-4330-b5f6-c58eb893715a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:55.953826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283b5ee-a8c7-47fa-8dc5-71b57a67070b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:55.954888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283b5ee-a8c7-47fa-8dc5-71b57a67070b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:55.955294 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283b5ee-a8c7-47fa-8dc5-71b57a67070b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:55.955783 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283b5ee-a8c7-47fa-8dc5-71b57a67070b admin admin] PATH_INFO: `/v3/projects/5038133430d84bd48c6da9ca734737cf` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:55.957812 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283b5ee-a8c7-47fa-8dc5-71b57a67070b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5038133430d84bd48c6da9ca734737cf)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:55.960386 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283b5ee-a8c7-47fa-8dc5-71b57a67070b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:56.049772 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1283b5ee-a8c7-47fa-8dc5-71b57a67070b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5038133430d84bd48c6da9ca734737cf', 'request_id': 'req-1283b5ee-a8c7-47fa-8dc5-71b57a67070b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:56.058979 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1283b5ee-a8c7-47fa-8dc5-71b57a67070b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5038133430d84bd48c6da9ca734737cf', 'request_id': 'req-1283b5ee-a8c7-47fa-8dc5-71b57a67070b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:56.067716 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1283b5ee-a8c7-47fa-8dc5-71b57a67070b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5038133430d84bd48c6da9ca734737cf', 'request_id': 'req-1283b5ee-a8c7-47fa-8dc5-71b57a67070b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:56.096114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1283b5ee-a8c7-47fa-8dc5-71b57a67070b admin admin] The token cache is being invalidate because project 5038133430d84bd48c6da9ca734737cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:25:56.096542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1283b5ee-a8c7-47fa-8dc5-71b57a67070b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:56.097629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1283b5ee-a8c7-47fa-8dc5-71b57a67070b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:25:56.099300 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1147/2312] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:25:55 2022] DELETE /identity/v3/projects/5038133430d84bd48c6da9ca734737cf => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:56.105528 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f27110-1292-478e-9d89-d912c6cdcbad None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:56.105888 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f27110-1292-478e-9d89-d912c6cdcbad None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:56.109007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f27110-1292-478e-9d89-d912c6cdcbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:56.169576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-53f27110-1292-478e-9d89-d912c6cdcbad None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:56.171871 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-53f27110-1292-478e-9d89-d912c6cdcbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:56.246891 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:56.286662 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1166/2313] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:56.295057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba58ce-6d76-4371-8184-fd989ce85c45 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:56.295424 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba58ce-6d76-4371-8184-fd989ce85c45 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:56.295798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba58ce-6d76-4371-8184-fd989ce85c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:56.323003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:56.329047 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-08ba58ce-6d76-4371-8184-fd989ce85c45 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:56.331347 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-08ba58ce-6d76-4371-8184-fd989ce85c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:56.341224 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1148/2314] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:56.345623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f27110-1292-478e-9d89-d912c6cdcbad None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:56.359873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f27110-1292-478e-9d89-d912c6cdcbad None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:56.365694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a1c74-e140-4774-b54d-1e12fb05b903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:56.367440 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a1c74-e140-4774-b54d-1e12fb05b903 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:56.368081 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a1c74-e140-4774-b54d-1e12fb05b903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:56.368546 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a1c74-e140-4774-b54d-1e12fb05b903 admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:56.369454 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a1c74-e140-4774-b54d-1e12fb05b903 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:25:56.369584 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a1c74-e140-4774-b54d-1e12fb05b903 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:56.375598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a1c74-e140-4774-b54d-1e12fb05b903 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:56.440104 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1167/2315] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:25:56.468747 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba58ce-6d76-4371-8184-fd989ce85c45 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:56.484495 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba58ce-6d76-4371-8184-fd989ce85c45 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:56.499084 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9d3d8-0003-4de2-a5d8-bf29ad6d2e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:56.500212 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9d3d8-0003-4de2-a5d8-bf29ad6d2e64 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:56.500718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9d3d8-0003-4de2-a5d8-bf29ad6d2e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:56.501200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9d3d8-0003-4de2-a5d8-bf29ad6d2e64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:56.502064 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9d3d8-0003-4de2-a5d8-bf29ad6d2e64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:56.504267 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9d3d8-0003-4de2-a5d8-bf29ad6d2e64 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:56.584765 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1149/2316] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:56 2022] POST /identity/v3/projects => generated 381 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:25:56.588980 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a1c74-e140-4774-b54d-1e12fb05b903 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:56.602661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a1c74-e140-4774-b54d-1e12fb05b903 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:56.683747 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58003bb-11f0-4db4-8045-89936de17679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:56.684479 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58003bb-11f0-4db4-8045-89936de17679 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:56.684836 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58003bb-11f0-4db4-8045-89936de17679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:56.685185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58003bb-11f0-4db4-8045-89936de17679 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:56.685895 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58003bb-11f0-4db4-8045-89936de17679 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:56.687995 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58003bb-11f0-4db4-8045-89936de17679 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:56.718557 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1168/2317] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:56 2022] POST /identity/v3/users => generated 291 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:25:56.719935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9d3d8-0003-4de2-a5d8-bf29ad6d2e64 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:56.731135 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9d3d8-0003-4de2-a5d8-bf29ad6d2e64 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:56.738401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc272b-529d-4580-bf3b-12d6e6a7e7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:56.739008 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc272b-529d-4580-bf3b-12d6e6a7e7ef admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:56.739287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc272b-529d-4580-bf3b-12d6e6a7e7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:56.739560 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc272b-529d-4580-bf3b-12d6e6a7e7ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:56.740267 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc272b-529d-4580-bf3b-12d6e6a7e7ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:56.741990 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc272b-529d-4580-bf3b-12d6e6a7e7ef admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:56.753157 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1150/2318] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:56.757148 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58003bb-11f0-4db4-8045-89936de17679 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:56.769054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58003bb-11f0-4db4-8045-89936de17679 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:56.774453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc1d58-820d-49f2-b145-8f6c59cfcb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:56.775057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc1d58-820d-49f2-b145-8f6c59cfcb44 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:56.775335 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc1d58-820d-49f2-b145-8f6c59cfcb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:56.775604 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc1d58-820d-49f2-b145-8f6c59cfcb44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:56.776351 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc1d58-820d-49f2-b145-8f6c59cfcb44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:56.777971 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc1d58-820d-49f2-b145-8f6c59cfcb44 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:56.791467 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1169/2319] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:56.795314 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc272b-529d-4580-bf3b-12d6e6a7e7ef admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:56.807706 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc272b-529d-4580-bf3b-12d6e6a7e7ef admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:56.813216 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187309c1-a998-4978-8816-ced554781531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:56.813865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187309c1-a998-4978-8816-ced554781531 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:56.814142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187309c1-a998-4978-8816-ced554781531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:56.814418 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187309c1-a998-4978-8816-ced554781531 admin admin] PATH_INFO: `/v3/projects/a37b4fa4c2cf4b50a23f46ef357ef9c5/users/fc67628215134849bc79c84d618c99a8/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:56.831939 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187309c1-a998-4978-8816-ced554781531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a37b4fa4c2cf4b50a23f46ef357ef9c5, user_id=fc67628215134849bc79c84d618c99a8, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:56.835039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187309c1-a998-4978-8816-ced554781531 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:56.857575 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1151/2320] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:56 2022] PUT /identity/v3/projects/a37b4fa4c2cf4b50a23f46ef357ef9c5/users/fc67628215134849bc79c84d618c99a8/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:56.869946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbb2db-dc50-4314-8d90-3086b97ab6a2 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:56.870275 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbb2db-dc50-4314-8d90-3086b97ab6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:56.870497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbb2db-dc50-4314-8d90-3086b97ab6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:56.901279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-e5dbb2db-dc50-4314-8d90-3086b97ab6a2 None None] MFA Rules not processed for user `fc67628215134849bc79c84d618c99a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:56.902238 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e5dbb2db-dc50-4314-8d90-3086b97ab6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:56.998927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1170/2321] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:57.000007 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187309c1-a998-4978-8816-ced554781531 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:57.079770 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187309c1-a998-4978-8816-ced554781531 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:57.085053 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564bf84-bbd2-49c7-8315-1931ad3e188f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:57.085670 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564bf84-bbd2-49c7-8315-1931ad3e188f admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:57.085951 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564bf84-bbd2-49c7-8315-1931ad3e188f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:57.086310 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564bf84-bbd2-49c7-8315-1931ad3e188f admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:57.087350 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564bf84-bbd2-49c7-8315-1931ad3e188f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:57.090172 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564bf84-bbd2-49c7-8315-1931ad3e188f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:57.128880 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1152/2322] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:25:56 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:25:57.133935 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbb2db-dc50-4314-8d90-3086b97ab6a2 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:57.146697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbb2db-dc50-4314-8d90-3086b97ab6a2 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:57.191608 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:57.239710 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c9420-4df4-4746-a732-46b3185f1579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:57.241678 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c9420-4df4-4746-a732-46b3185f1579 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:57.242121 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c9420-4df4-4746-a732-46b3185f1579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:57.242591 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c9420-4df4-4746-a732-46b3185f1579 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:57.243405 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c9420-4df4-4746-a732-46b3185f1579 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:57.246186 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c9420-4df4-4746-a732-46b3185f1579 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:57.347182 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1171/2323] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:25:57 2022] POST /identity/v3/users => generated 298 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:25:57.348518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564bf84-bbd2-49c7-8315-1931ad3e188f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:57.362175 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564bf84-bbd2-49c7-8315-1931ad3e188f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:57.376169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfed8b-1987-4683-b28a-b6ee9079f6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:57.377203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfed8b-1987-4683-b28a-b6ee9079f6c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:57.377203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfed8b-1987-4683-b28a-b6ee9079f6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:57.377428 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfed8b-1987-4683-b28a-b6ee9079f6c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:57.387904 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccfed8b-1987-4683-b28a-b6ee9079f6c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:57.387904 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccfed8b-1987-4683-b28a-b6ee9079f6c2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:57.413448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1153/2324] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:57.427664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c9420-4df4-4746-a732-46b3185f1579 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:57.441020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c9420-4df4-4746-a732-46b3185f1579 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:57.463637 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8fece-fc8c-4133-a95f-ceb54cddeeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:57.464359 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8fece-fc8c-4133-a95f-ceb54cddeeb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:57.464643 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8fece-fc8c-4133-a95f-ceb54cddeeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:57.464920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8fece-fc8c-4133-a95f-ceb54cddeeb6 admin admin] PATH_INFO: `/v3/projects/fba9470f652643a39ccfea468aae2043/users/96e4b17454444f82ab223d0be3bef3cd/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:57.493400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8fece-fc8c-4133-a95f-ceb54cddeeb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fba9470f652643a39ccfea468aae2043, user_id=96e4b17454444f82ab223d0be3bef3cd, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:57.496252 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8fece-fc8c-4133-a95f-ceb54cddeeb6 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:57.519571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1172/2325] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:57 2022] PUT /identity/v3/projects/fba9470f652643a39ccfea468aae2043/users/96e4b17454444f82ab223d0be3bef3cd/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:25:57.520940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfed8b-1987-4683-b28a-b6ee9079f6c2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:57.534901 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfed8b-1987-4683-b28a-b6ee9079f6c2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:57.638903 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33098c23-7ca8-41fc-ab4e-e4355b987505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:57.639145 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33098c23-7ca8-41fc-ab4e-e4355b987505 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:57.639663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33098c23-7ca8-41fc-ab4e-e4355b987505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:57.640366 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33098c23-7ca8-41fc-ab4e-e4355b987505 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:57.641204 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33098c23-7ca8-41fc-ab4e-e4355b987505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:57.643076 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33098c23-7ca8-41fc-ab4e-e4355b987505 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:57.662047 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1154/2326] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:25:57.663576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8fece-fc8c-4133-a95f-ceb54cddeeb6 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:57.675735 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8fece-fc8c-4133-a95f-ceb54cddeeb6 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:57.681680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc413f9-c7c3-4ac4-8765-b006d290e8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:57.682348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc413f9-c7c3-4ac4-8765-b006d290e8ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:57.682658 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc413f9-c7c3-4ac4-8765-b006d290e8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:57.682928 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc413f9-c7c3-4ac4-8765-b006d290e8ec admin admin] PATH_INFO: `/v3/projects/fba9470f652643a39ccfea468aae2043/users/96e4b17454444f82ab223d0be3bef3cd/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:57.685012 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc413f9-c7c3-4ac4-8765-b006d290e8ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=fba9470f652643a39ccfea468aae2043, user_id=96e4b17454444f82ab223d0be3bef3cd, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:57.687871 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc413f9-c7c3-4ac4-8765-b006d290e8ec admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:57.700481 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1173/2327] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:25:57 2022] PUT /identity/v3/projects/fba9470f652643a39ccfea468aae2043/users/96e4b17454444f82ab223d0be3bef3cd/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:25:57.712175 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d61d7-67f0-4532-a6dd-13307eef0a42 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:57.712712 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d61d7-67f0-4532-a6dd-13307eef0a42 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:57.712712 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d61d7-67f0-4532-a6dd-13307eef0a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:57.739340 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:25:57.743636 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f48d61d7-67f0-4532-a6dd-13307eef0a42 None None] MFA Rules not processed for user `96e4b17454444f82ab223d0be3bef3cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:25:57.745380 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f48d61d7-67f0-4532-a6dd-13307eef0a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:57.823321 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1155/2328] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:25:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:25:57.837712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc413f9-c7c3-4ac4-8765-b006d290e8ec admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:57.839480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-fbc413f9-c7c3-4ac4-8765-b006d290e8ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:25:57.915940 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc413f9-c7c3-4ac4-8765-b006d290e8ec admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:57.919782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320fa153-8de7-48f2-a85a-07fa198ff387 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:57.920535 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fa153-8de7-48f2-a85a-07fa198ff387 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:57.920919 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fa153-8de7-48f2-a85a-07fa198ff387 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:57.921316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fa153-8de7-48f2-a85a-07fa198ff387 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:57.933812 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320fa153-8de7-48f2-a85a-07fa198ff387 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:57.935547 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320fa153-8de7-48f2-a85a-07fa198ff387 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:57.952415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1174/2329] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:57 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:25:58.075329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d61d7-67f0-4532-a6dd-13307eef0a42 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:25:58.087110 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d61d7-67f0-4532-a6dd-13307eef0a42 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:25:58.092130 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee9089-bcca-4a96-a4e5-4faf0bcae926 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:25:58.093253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee9089-bcca-4a96-a4e5-4faf0bcae926 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:25:58.093721 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee9089-bcca-4a96-a4e5-4faf0bcae926 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:25:58.094052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee9089-bcca-4a96-a4e5-4faf0bcae926 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:25:58.166384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee9089-bcca-4a96-a4e5-4faf0bcae926 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:25:58.168224 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee9089-bcca-4a96-a4e5-4faf0bcae926 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:25:58.191123 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1156/2330] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:25:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:01.028594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320fa153-8de7-48f2-a85a-07fa198ff387 service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:01.106107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320fa153-8de7-48f2-a85a-07fa198ff387 service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:01.111041 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8875d-4d09-45fb-b378-733de0d234a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:01.111793 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8875d-4d09-45fb-b378-733de0d234a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:01.112090 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8875d-4d09-45fb-b378-733de0d234a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:01.112374 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8875d-4d09-45fb-b378-733de0d234a5 admin admin] PATH_INFO: `/v3/users/fc67628215134849bc79c84d618c99a8` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:01.113790 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8875d-4d09-45fb-b378-733de0d234a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc67628215134849bc79c84d618c99a8)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:01.115874 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8875d-4d09-45fb-b378-733de0d234a5 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:01.202040 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:01.227494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-74c8875d-4d09-45fb-b378-733de0d234a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc67628215134849bc79c84d618c99a8'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:01.228096 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-74c8875d-4d09-45fb-b378-733de0d234a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc67628215134849bc79c84d618c99a8'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:01.229078 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-74c8875d-4d09-45fb-b378-733de0d234a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc67628215134849bc79c84d618c99a8'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:01.236826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-74c8875d-4d09-45fb-b378-733de0d234a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc67628215134849bc79c84d618c99a8'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:01.268705 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-74c8875d-4d09-45fb-b378-733de0d234a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc67628215134849bc79c84d618c99a8'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:01.285304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1175/2331] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:01 2022] DELETE /identity/v3/users/fc67628215134849bc79c84d618c99a8 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:01.290260 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750029b-dcb2-4d67-9438-bc5ecfdd8038 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:01.291230 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750029b-dcb2-4d67-9438-bc5ecfdd8038 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:01.291857 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750029b-dcb2-4d67-9438-bc5ecfdd8038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:01.319202 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-1750029b-dcb2-4d67-9438-bc5ecfdd8038 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:01.320616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1750029b-dcb2-4d67-9438-bc5ecfdd8038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:01.413716 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1157/2332] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:01.424710 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8875d-4d09-45fb-b378-733de0d234a5 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:01.427066 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-74c8875d-4d09-45fb-b378-733de0d234a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:01.502947 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8875d-4d09-45fb-b378-733de0d234a5 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:01.506750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844f789-0598-4427-9755-396259d76c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:01.507361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844f789-0598-4427-9755-396259d76c88 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:01.507638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844f789-0598-4427-9755-396259d76c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:01.507942 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844f789-0598-4427-9755-396259d76c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:01.521244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844f789-0598-4427-9755-396259d76c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:01.522721 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844f789-0598-4427-9755-396259d76c88 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:01.533174 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1176/2333] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:01.876725 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750029b-dcb2-4d67-9438-bc5ecfdd8038 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:01.877935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1750029b-dcb2-4d67-9438-bc5ecfdd8038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:01.906895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750029b-dcb2-4d67-9438-bc5ecfdd8038 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:01.925383 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c190c0-6df0-449d-a4d3-93d48d715f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:01.926051 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c190c0-6df0-449d-a4d3-93d48d715f59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:01.926327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c190c0-6df0-449d-a4d3-93d48d715f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:01.926595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c190c0-6df0-449d-a4d3-93d48d715f59 admin admin] PATH_INFO: `/v3/projects/a37b4fa4c2cf4b50a23f46ef357ef9c5` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:01.928475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c190c0-6df0-449d-a4d3-93d48d715f59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a37b4fa4c2cf4b50a23f46ef357ef9c5)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:01.930423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c190c0-6df0-449d-a4d3-93d48d715f59 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:01.974073 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:01.988692 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-82c190c0-6df0-449d-a4d3-93d48d715f59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a37b4fa4c2cf4b50a23f46ef357ef9c5', 'request_id': 'req-82c190c0-6df0-449d-a4d3-93d48d715f59'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:01.998111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-82c190c0-6df0-449d-a4d3-93d48d715f59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a37b4fa4c2cf4b50a23f46ef357ef9c5', 'request_id': 'req-82c190c0-6df0-449d-a4d3-93d48d715f59'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:02.006237 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-82c190c0-6df0-449d-a4d3-93d48d715f59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a37b4fa4c2cf4b50a23f46ef357ef9c5', 'request_id': 'req-82c190c0-6df0-449d-a4d3-93d48d715f59'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:02.020782 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-82c190c0-6df0-449d-a4d3-93d48d715f59 admin admin] The token cache is being invalidate because project a37b4fa4c2cf4b50a23f46ef357ef9c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:02.021247 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-82c190c0-6df0-449d-a4d3-93d48d715f59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:02.022651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-82c190c0-6df0-449d-a4d3-93d48d715f59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:02.027317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1158/2334] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:01 2022] DELETE /identity/v3/projects/a37b4fa4c2cf4b50a23f46ef357ef9c5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:02.028653 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844f789-0598-4427-9755-396259d76c88 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:02.039609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844f789-0598-4427-9755-396259d76c88 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:02.109128 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7232ec-019f-4ae2-a153-ad3ff0e588b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:02.110007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7232ec-019f-4ae2-a153-ad3ff0e588b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:02.110357 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7232ec-019f-4ae2-a153-ad3ff0e588b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:02.110640 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7232ec-019f-4ae2-a153-ad3ff0e588b9 admin admin] PATH_INFO: `/v3/users/96e4b17454444f82ab223d0be3bef3cd` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:02.113346 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7232ec-019f-4ae2-a153-ad3ff0e588b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96e4b17454444f82ab223d0be3bef3cd)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:02.115513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7232ec-019f-4ae2-a153-ad3ff0e588b9 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:02.180304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4d7232ec-019f-4ae2-a153-ad3ff0e588b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96e4b17454444f82ab223d0be3bef3cd'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:02.182820 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4d7232ec-019f-4ae2-a153-ad3ff0e588b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96e4b17454444f82ab223d0be3bef3cd'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:02.183539 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4d7232ec-019f-4ae2-a153-ad3ff0e588b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96e4b17454444f82ab223d0be3bef3cd'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:02.190756 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:02.193786 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4d7232ec-019f-4ae2-a153-ad3ff0e588b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96e4b17454444f82ab223d0be3bef3cd'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:02.210020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4d7232ec-019f-4ae2-a153-ad3ff0e588b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96e4b17454444f82ab223d0be3bef3cd'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:02.225205 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1177/2335] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:02 2022] DELETE /identity/v3/users/96e4b17454444f82ab223d0be3bef3cd => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:02.723930 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c190c0-6df0-449d-a4d3-93d48d715f59 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:02.726484 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-82c190c0-6df0-449d-a4d3-93d48d715f59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:02.745946 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c190c0-6df0-449d-a4d3-93d48d715f59 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:02.805986 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05bbce-c8cc-451f-8dea-2d0e8413b506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:02.806562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05bbce-c8cc-451f-8dea-2d0e8413b506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:02.806841 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05bbce-c8cc-451f-8dea-2d0e8413b506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:02.807147 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05bbce-c8cc-451f-8dea-2d0e8413b506 admin admin] PATH_INFO: `/v3/projects/fba9470f652643a39ccfea468aae2043` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:02.808408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05bbce-c8cc-451f-8dea-2d0e8413b506 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fba9470f652643a39ccfea468aae2043)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:02.810371 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05bbce-c8cc-451f-8dea-2d0e8413b506 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:02.858260 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4f05bbce-c8cc-451f-8dea-2d0e8413b506 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fba9470f652643a39ccfea468aae2043', 'request_id': 'req-4f05bbce-c8cc-451f-8dea-2d0e8413b506'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:02.864520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4f05bbce-c8cc-451f-8dea-2d0e8413b506 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fba9470f652643a39ccfea468aae2043', 'request_id': 'req-4f05bbce-c8cc-451f-8dea-2d0e8413b506'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:02.870633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4f05bbce-c8cc-451f-8dea-2d0e8413b506 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fba9470f652643a39ccfea468aae2043', 'request_id': 'req-4f05bbce-c8cc-451f-8dea-2d0e8413b506'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:02.883546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4f05bbce-c8cc-451f-8dea-2d0e8413b506 admin admin] The token cache is being invalidate because project fba9470f652643a39ccfea468aae2043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:02.883877 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4f05bbce-c8cc-451f-8dea-2d0e8413b506 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:02.884538 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4f05bbce-c8cc-451f-8dea-2d0e8413b506 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:02.886020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1159/2336] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:02 2022] DELETE /identity/v3/projects/fba9470f652643a39ccfea468aae2043 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:02.890878 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57071e02-3dcd-4eb6-989e-39993d8cd1e8 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:02.891167 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57071e02-3dcd-4eb6-989e-39993d8cd1e8 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:02.891445 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57071e02-3dcd-4eb6-989e-39993d8cd1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:02.911455 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-57071e02-3dcd-4eb6-989e-39993d8cd1e8 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:02.912762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-57071e02-3dcd-4eb6-989e-39993d8cd1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:02.973424 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1178/2337] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:02.981889 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f51588-554d-4e86-b3d4-925d8ba3b429 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:02.982198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f51588-554d-4e86-b3d4-925d8ba3b429 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:02.982478 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f51588-554d-4e86-b3d4-925d8ba3b429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:03.002144 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-c9f51588-554d-4e86-b3d4-925d8ba3b429 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:03.003360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c9f51588-554d-4e86-b3d4-925d8ba3b429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:03.012361 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1160/2338] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:03.016891 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57071e02-3dcd-4eb6-989e-39993d8cd1e8 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:03.027152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57071e02-3dcd-4eb6-989e-39993d8cd1e8 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:03.031797 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cfdf6-a9ee-450d-9ec1-e8f5482fef10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:03.032393 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cfdf6-a9ee-450d-9ec1-e8f5482fef10 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:03.032655 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cfdf6-a9ee-450d-9ec1-e8f5482fef10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:03.032929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cfdf6-a9ee-450d-9ec1-e8f5482fef10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:03.033581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86cfdf6-a9ee-450d-9ec1-e8f5482fef10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:26:03.034227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86cfdf6-a9ee-450d-9ec1-e8f5482fef10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:03.035487 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86cfdf6-a9ee-450d-9ec1-e8f5482fef10 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:03.047094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1179/2339] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:26:03.051403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f51588-554d-4e86-b3d4-925d8ba3b429 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:03.061752 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f51588-554d-4e86-b3d4-925d8ba3b429 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:03.066433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e6972-69cd-45ac-b65e-8eb372fd500e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:03.067031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e6972-69cd-45ac-b65e-8eb372fd500e admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:03.067313 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e6972-69cd-45ac-b65e-8eb372fd500e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:03.067594 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e6972-69cd-45ac-b65e-8eb372fd500e admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:03.068278 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4e6972-69cd-45ac-b65e-8eb372fd500e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:03.070863 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4e6972-69cd-45ac-b65e-8eb372fd500e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:03.099381 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1161/2340] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:03 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:26:03.103543 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cfdf6-a9ee-450d-9ec1-e8f5482fef10 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:03.113591 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cfdf6-a9ee-450d-9ec1-e8f5482fef10 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:03.171593 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72720e51-5e29-48d2-a76f-93ff2c0a70ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:03.172226 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72720e51-5e29-48d2-a76f-93ff2c0a70ab admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:03.172507 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72720e51-5e29-48d2-a76f-93ff2c0a70ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:03.172777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72720e51-5e29-48d2-a76f-93ff2c0a70ab admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:03.173411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72720e51-5e29-48d2-a76f-93ff2c0a70ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:03.175379 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72720e51-5e29-48d2-a76f-93ff2c0a70ab admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:03.196117 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1180/2341] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:03 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:03.200175 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e6972-69cd-45ac-b65e-8eb372fd500e admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:03.210874 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e6972-69cd-45ac-b65e-8eb372fd500e admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:03.215567 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde15029-22b0-4653-814d-3ce14835149b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:03.216190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde15029-22b0-4653-814d-3ce14835149b admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:03.216475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde15029-22b0-4653-814d-3ce14835149b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:03.216749 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde15029-22b0-4653-814d-3ce14835149b admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:03.217395 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde15029-22b0-4653-814d-3ce14835149b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:03.219030 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde15029-22b0-4653-814d-3ce14835149b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:03.229757 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1162/2342] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:03.233715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72720e51-5e29-48d2-a76f-93ff2c0a70ab admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:03.246228 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72720e51-5e29-48d2-a76f-93ff2c0a70ab admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:03.251008 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda7729-22fe-48a1-9822-b6a9343febf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:03.251599 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda7729-22fe-48a1-9822-b6a9343febf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:03.251927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda7729-22fe-48a1-9822-b6a9343febf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:03.252196 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda7729-22fe-48a1-9822-b6a9343febf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:03.252832 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda7729-22fe-48a1-9822-b6a9343febf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:03.254427 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda7729-22fe-48a1-9822-b6a9343febf9 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:03.265427 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1181/2343] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:03.269713 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde15029-22b0-4653-814d-3ce14835149b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:03.280065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde15029-22b0-4653-814d-3ce14835149b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:03.284939 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac728f-56b7-4dde-86b5-77c164dffbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:03.285618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac728f-56b7-4dde-86b5-77c164dffbed admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:03.285900 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac728f-56b7-4dde-86b5-77c164dffbed admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:03.286176 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac728f-56b7-4dde-86b5-77c164dffbed admin admin] PATH_INFO: `/v3/projects/073c66701c814c93b2824cdca9c5300b/users/196926517b4c4163b51e9ecfd83e2e1c/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:03.296141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:03.310719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac728f-56b7-4dde-86b5-77c164dffbed admin admin] RBAC: Authorizing `identity:create_grant(project_id=073c66701c814c93b2824cdca9c5300b, user_id=196926517b4c4163b51e9ecfd83e2e1c, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:03.314039 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac728f-56b7-4dde-86b5-77c164dffbed admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:03.327685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1163/2344] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:03 2022] PUT /identity/v3/projects/073c66701c814c93b2824cdca9c5300b/users/196926517b4c4163b51e9ecfd83e2e1c/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:03.339307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5372c-283b-4430-8e2e-91e7c0f9ba05 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:03.339597 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5372c-283b-4430-8e2e-91e7c0f9ba05 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:03.339917 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5372c-283b-4430-8e2e-91e7c0f9ba05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:03.370411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-23e5372c-283b-4430-8e2e-91e7c0f9ba05 None None] MFA Rules not processed for user `196926517b4c4163b51e9ecfd83e2e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:03.371646 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-23e5372c-283b-4430-8e2e-91e7c0f9ba05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:03.397980 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:03.439206 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1182/2345] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:26:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:03.443498 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac728f-56b7-4dde-86b5-77c164dffbed admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:03.505616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac728f-56b7-4dde-86b5-77c164dffbed admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:03.513983 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627a7d5-e5c4-4b03-b577-54231ec2caf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:03.514945 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627a7d5-e5c4-4b03-b577-54231ec2caf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:03.515445 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627a7d5-e5c4-4b03-b577-54231ec2caf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:03.515990 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627a7d5-e5c4-4b03-b577-54231ec2caf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:03.517170 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c627a7d5-e5c4-4b03-b577-54231ec2caf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:03.519261 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c627a7d5-e5c4-4b03-b577-54231ec2caf1 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:03.547388 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1164/2346] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:03 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:03.551197 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5372c-283b-4430-8e2e-91e7c0f9ba05 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:03.562083 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5372c-283b-4430-8e2e-91e7c0f9ba05 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:03.619928 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d98014-48c6-47a8-93a1-c26c855c5b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:03.620545 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d98014-48c6-47a8-93a1-c26c855c5b11 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:03.620889 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d98014-48c6-47a8-93a1-c26c855c5b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:03.621161 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d98014-48c6-47a8-93a1-c26c855c5b11 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:03.621748 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d98014-48c6-47a8-93a1-c26c855c5b11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:03.623755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d98014-48c6-47a8-93a1-c26c855c5b11 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:03.647970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1183/2347] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:03 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:03.651594 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627a7d5-e5c4-4b03-b577-54231ec2caf1 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:03.667876 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627a7d5-e5c4-4b03-b577-54231ec2caf1 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:03.674144 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c47f1-a6aa-449f-9d99-6069f78ff94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:03.675062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c47f1-a6aa-449f-9d99-6069f78ff94e admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:03.675333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c47f1-a6aa-449f-9d99-6069f78ff94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:03.675630 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c47f1-a6aa-449f-9d99-6069f78ff94e admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:03.676299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c47f1-a6aa-449f-9d99-6069f78ff94e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:03.679199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c47f1-a6aa-449f-9d99-6069f78ff94e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:03.699675 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1165/2348] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:03.702500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d98014-48c6-47a8-93a1-c26c855c5b11 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:03.718248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d98014-48c6-47a8-93a1-c26c855c5b11 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:03.723729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824f580-36dc-42ee-a90c-f191d5db9259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:03.725019 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824f580-36dc-42ee-a90c-f191d5db9259 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:03.725019 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824f580-36dc-42ee-a90c-f191d5db9259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:03.725359 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824f580-36dc-42ee-a90c-f191d5db9259 admin admin] PATH_INFO: `/v3/projects/65c81473013c47a2b42033242f11e1a1/users/6d0bdef4b86342e7805a7dcf380ea313/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:03.755274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824f580-36dc-42ee-a90c-f191d5db9259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65c81473013c47a2b42033242f11e1a1, user_id=6d0bdef4b86342e7805a7dcf380ea313, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:03.760286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824f580-36dc-42ee-a90c-f191d5db9259 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:03.791160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1184/2349] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:03 2022] PUT /identity/v3/projects/65c81473013c47a2b42033242f11e1a1/users/6d0bdef4b86342e7805a7dcf380ea313/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:03.792499 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c47f1-a6aa-449f-9d99-6069f78ff94e admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:03.806017 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c47f1-a6aa-449f-9d99-6069f78ff94e admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:03.913570 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20173a53-d037-4ed7-b0c8-94d70cbdfe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:03.914186 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20173a53-d037-4ed7-b0c8-94d70cbdfe2d admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:03.914469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20173a53-d037-4ed7-b0c8-94d70cbdfe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:03.915206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20173a53-d037-4ed7-b0c8-94d70cbdfe2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:03.915901 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20173a53-d037-4ed7-b0c8-94d70cbdfe2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:03.917694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20173a53-d037-4ed7-b0c8-94d70cbdfe2d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:03.931868 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1166/2350] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:03.933693 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824f580-36dc-42ee-a90c-f191d5db9259 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:03.945713 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824f580-36dc-42ee-a90c-f191d5db9259 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:03.950521 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ebf71-0dc3-4ad4-abbd-12d309e8cd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:03.951171 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ebf71-0dc3-4ad4-abbd-12d309e8cd63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:03.951452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ebf71-0dc3-4ad4-abbd-12d309e8cd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:03.951727 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ebf71-0dc3-4ad4-abbd-12d309e8cd63 admin admin] PATH_INFO: `/v3/projects/65c81473013c47a2b42033242f11e1a1/users/6d0bdef4b86342e7805a7dcf380ea313/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:03.953715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ebf71-0dc3-4ad4-abbd-12d309e8cd63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65c81473013c47a2b42033242f11e1a1, user_id=6d0bdef4b86342e7805a7dcf380ea313, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:03.956382 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ebf71-0dc3-4ad4-abbd-12d309e8cd63 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:03.968762 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1185/2351] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:03 2022] PUT /identity/v3/projects/65c81473013c47a2b42033242f11e1a1/users/6d0bdef4b86342e7805a7dcf380ea313/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:26:03.979017 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a7d9b-e347-440f-aaf9-4ce2dab3e0e2 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:03.979307 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a7d9b-e347-440f-aaf9-4ce2dab3e0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:03.979593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a7d9b-e347-440f-aaf9-4ce2dab3e0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:04.008951 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-cb0a7d9b-e347-440f-aaf9-4ce2dab3e0e2 None None] MFA Rules not processed for user `6d0bdef4b86342e7805a7dcf380ea313`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:04.010312 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-cb0a7d9b-e347-440f-aaf9-4ce2dab3e0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:04.069521 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:04.105087 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1167/2352] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:26:03 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:04.116585 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ebf71-0dc3-4ad4-abbd-12d309e8cd63 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:04.120912 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5d2ebf71-0dc3-4ad4-abbd-12d309e8cd63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:04.205258 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ebf71-0dc3-4ad4-abbd-12d309e8cd63 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:04.213388 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32c414-7238-4736-942a-b399468b4c7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:04.214587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32c414-7238-4736-942a-b399468b4c7c service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:04.215318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32c414-7238-4736-942a-b399468b4c7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:04.216337 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32c414-7238-4736-942a-b399468b4c7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:04.228973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c32c414-7238-4736-942a-b399468b4c7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:04.231069 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c32c414-7238-4736-942a-b399468b4c7c service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:04.245775 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1186/2353] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:04 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:04.971123 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a7d9b-e347-440f-aaf9-4ce2dab3e0e2 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:05.081832 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a7d9b-e347-440f-aaf9-4ce2dab3e0e2 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:05.087160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c93d85-05fa-4178-9ae5-5516d4881e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:05.087924 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c93d85-05fa-4178-9ae5-5516d4881e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:05.088217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c93d85-05fa-4178-9ae5-5516d4881e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:05.088494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c93d85-05fa-4178-9ae5-5516d4881e9f admin admin] PATH_INFO: `/v3/users/196926517b4c4163b51e9ecfd83e2e1c` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:05.089647 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c93d85-05fa-4178-9ae5-5516d4881e9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=196926517b4c4163b51e9ecfd83e2e1c)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:05.091939 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c93d85-05fa-4178-9ae5-5516d4881e9f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:05.170320 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-42c93d85-05fa-4178-9ae5-5516d4881e9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '196926517b4c4163b51e9ecfd83e2e1c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:05.181144 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-42c93d85-05fa-4178-9ae5-5516d4881e9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '196926517b4c4163b51e9ecfd83e2e1c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:05.181882 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-42c93d85-05fa-4178-9ae5-5516d4881e9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '196926517b4c4163b51e9ecfd83e2e1c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:05.182520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-42c93d85-05fa-4178-9ae5-5516d4881e9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '196926517b4c4163b51e9ecfd83e2e1c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:05.206519 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-42c93d85-05fa-4178-9ae5-5516d4881e9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '196926517b4c4163b51e9ecfd83e2e1c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:05.235535 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1168/2354] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:04 2022] DELETE /identity/v3/users/196926517b4c4163b51e9ecfd83e2e1c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:05.241623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eacb78-2dd8-4e6f-894b-77cfe71a0fa7 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:05.242145 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eacb78-2dd8-4e6f-894b-77cfe71a0fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:05.242360 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eacb78-2dd8-4e6f-894b-77cfe71a0fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:05.263173 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-23eacb78-2dd8-4e6f-894b-77cfe71a0fa7 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:05.264916 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-23eacb78-2dd8-4e6f-894b-77cfe71a0fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:05.286385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:05.356394 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1187/2355] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:26:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:05.366858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c93d85-05fa-4178-9ae5-5516d4881e9f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:05.369126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-42c93d85-05fa-4178-9ae5-5516d4881e9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:05.448200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c93d85-05fa-4178-9ae5-5516d4881e9f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:05.453851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c0f0a-5206-4193-b125-eb227f54f435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:05.455178 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c0f0a-5206-4193-b125-eb227f54f435 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:05.455790 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c0f0a-5206-4193-b125-eb227f54f435 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:05.456258 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c0f0a-5206-4193-b125-eb227f54f435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:05.468225 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c0f0a-5206-4193-b125-eb227f54f435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:05.469755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c0f0a-5206-4193-b125-eb227f54f435 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:05.481894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1169/2356] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:05.820077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eacb78-2dd8-4e6f-894b-77cfe71a0fa7 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:05.821306 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-23eacb78-2dd8-4e6f-894b-77cfe71a0fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:05.834561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eacb78-2dd8-4e6f-894b-77cfe71a0fa7 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:05.839420 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57cdd8-6089-4ac7-9100-5d3504cdc376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:05.840092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57cdd8-6089-4ac7-9100-5d3504cdc376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:05.840538 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57cdd8-6089-4ac7-9100-5d3504cdc376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:05.840941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57cdd8-6089-4ac7-9100-5d3504cdc376 admin admin] PATH_INFO: `/v3/projects/073c66701c814c93b2824cdca9c5300b` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:05.842454 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57cdd8-6089-4ac7-9100-5d3504cdc376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=073c66701c814c93b2824cdca9c5300b)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:05.844543 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57cdd8-6089-4ac7-9100-5d3504cdc376 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:05.898815 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-2e57cdd8-6089-4ac7-9100-5d3504cdc376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '073c66701c814c93b2824cdca9c5300b', 'request_id': 'req-2e57cdd8-6089-4ac7-9100-5d3504cdc376'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:05.907142 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-2e57cdd8-6089-4ac7-9100-5d3504cdc376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '073c66701c814c93b2824cdca9c5300b', 'request_id': 'req-2e57cdd8-6089-4ac7-9100-5d3504cdc376'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:05.915368 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-2e57cdd8-6089-4ac7-9100-5d3504cdc376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '073c66701c814c93b2824cdca9c5300b', 'request_id': 'req-2e57cdd8-6089-4ac7-9100-5d3504cdc376'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:05.928886 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-2e57cdd8-6089-4ac7-9100-5d3504cdc376 admin admin] The token cache is being invalidate because project 073c66701c814c93b2824cdca9c5300b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:05.929176 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-2e57cdd8-6089-4ac7-9100-5d3504cdc376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:05.929795 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-2e57cdd8-6089-4ac7-9100-5d3504cdc376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:05.931427 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1188/2357] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:05 2022] DELETE /identity/v3/projects/073c66701c814c93b2824cdca9c5300b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:05.935542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c0f0a-5206-4193-b125-eb227f54f435 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:05.942191 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:05.946332 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c0f0a-5206-4193-b125-eb227f54f435 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:06.008712 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29afa0ee-2786-43be-8fab-3b7eb4f76edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:06.009404 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afa0ee-2786-43be-8fab-3b7eb4f76edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:06.009608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afa0ee-2786-43be-8fab-3b7eb4f76edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:06.009996 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afa0ee-2786-43be-8fab-3b7eb4f76edc admin admin] PATH_INFO: `/v3/users/6d0bdef4b86342e7805a7dcf380ea313` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:06.011106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29afa0ee-2786-43be-8fab-3b7eb4f76edc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d0bdef4b86342e7805a7dcf380ea313)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:06.013047 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29afa0ee-2786-43be-8fab-3b7eb4f76edc admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:06.063568 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-29afa0ee-2786-43be-8fab-3b7eb4f76edc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d0bdef4b86342e7805a7dcf380ea313'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:06.072008 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-29afa0ee-2786-43be-8fab-3b7eb4f76edc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d0bdef4b86342e7805a7dcf380ea313'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:06.072692 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-29afa0ee-2786-43be-8fab-3b7eb4f76edc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d0bdef4b86342e7805a7dcf380ea313'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:06.073391 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-29afa0ee-2786-43be-8fab-3b7eb4f76edc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d0bdef4b86342e7805a7dcf380ea313'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:06.089368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-29afa0ee-2786-43be-8fab-3b7eb4f76edc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d0bdef4b86342e7805a7dcf380ea313'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:06.120075 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1170/2358] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:05 2022] DELETE /identity/v3/users/6d0bdef4b86342e7805a7dcf380ea313 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:06.467073 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57cdd8-6089-4ac7-9100-5d3504cdc376 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:06.468554 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2e57cdd8-6089-4ac7-9100-5d3504cdc376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:06.499426 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57cdd8-6089-4ac7-9100-5d3504cdc376 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:06.608737 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a69203-b3e9-4ed9-80f7-5207caa06191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:06.609378 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a69203-b3e9-4ed9-80f7-5207caa06191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:06.609659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a69203-b3e9-4ed9-80f7-5207caa06191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:06.609931 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a69203-b3e9-4ed9-80f7-5207caa06191 admin admin] PATH_INFO: `/v3/projects/65c81473013c47a2b42033242f11e1a1` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:06.611724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a69203-b3e9-4ed9-80f7-5207caa06191 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65c81473013c47a2b42033242f11e1a1)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:06.615046 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a69203-b3e9-4ed9-80f7-5207caa06191 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:06.687481 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:06.689112 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-09a69203-b3e9-4ed9-80f7-5207caa06191 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65c81473013c47a2b42033242f11e1a1', 'request_id': 'req-09a69203-b3e9-4ed9-80f7-5207caa06191'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:06.698884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-09a69203-b3e9-4ed9-80f7-5207caa06191 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65c81473013c47a2b42033242f11e1a1', 'request_id': 'req-09a69203-b3e9-4ed9-80f7-5207caa06191'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:06.706896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-09a69203-b3e9-4ed9-80f7-5207caa06191 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65c81473013c47a2b42033242f11e1a1', 'request_id': 'req-09a69203-b3e9-4ed9-80f7-5207caa06191'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:06.721653 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-09a69203-b3e9-4ed9-80f7-5207caa06191 admin admin] The token cache is being invalidate because project 65c81473013c47a2b42033242f11e1a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:06.722038 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-09a69203-b3e9-4ed9-80f7-5207caa06191 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:06.722755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-09a69203-b3e9-4ed9-80f7-5207caa06191 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:06.724561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1189/2359] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:06 2022] DELETE /identity/v3/projects/65c81473013c47a2b42033242f11e1a1 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:06.729935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113d4c5-99bc-4db2-891f-b97988d055fb None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:06.730437 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113d4c5-99bc-4db2-891f-b97988d055fb None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:06.730714 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113d4c5-99bc-4db2-891f-b97988d055fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:06.753303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-0113d4c5-99bc-4db2-891f-b97988d055fb None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:06.754534 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0113d4c5-99bc-4db2-891f-b97988d055fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:06.823139 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1171/2360] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:06.832001 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c4103-db0c-4fd1-8cea-a80ec0478ba0 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:06.832287 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c4103-db0c-4fd1-8cea-a80ec0478ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:06.832819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c4103-db0c-4fd1-8cea-a80ec0478ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:06.856934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-001c4103-db0c-4fd1-8cea-a80ec0478ba0 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:06.858286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-001c4103-db0c-4fd1-8cea-a80ec0478ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:06.868288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1190/2361] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:06.872359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113d4c5-99bc-4db2-891f-b97988d055fb None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:06.885076 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113d4c5-99bc-4db2-891f-b97988d055fb None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:06.892425 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6452f2-17dc-484a-bcca-443d2040aa2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:06.893373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6452f2-17dc-484a-bcca-443d2040aa2c admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:06.893766 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6452f2-17dc-484a-bcca-443d2040aa2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:06.894110 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6452f2-17dc-484a-bcca-443d2040aa2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:06.895082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6452f2-17dc-484a-bcca-443d2040aa2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:26:06.895473 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6452f2-17dc-484a-bcca-443d2040aa2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:06.898050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6452f2-17dc-484a-bcca-443d2040aa2c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:06.912318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1172/2362] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:26:06.917393 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c4103-db0c-4fd1-8cea-a80ec0478ba0 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:06.933523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c4103-db0c-4fd1-8cea-a80ec0478ba0 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:06.938736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c794f9-c4d5-431c-bcd5-3a1f88131c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:06.939375 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c794f9-c4d5-431c-bcd5-3a1f88131c78 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:06.939751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c794f9-c4d5-431c-bcd5-3a1f88131c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:06.940189 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c794f9-c4d5-431c-bcd5-3a1f88131c78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:06.940906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c794f9-c4d5-431c-bcd5-3a1f88131c78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:06.944061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c794f9-c4d5-431c-bcd5-3a1f88131c78 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:06.981235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1191/2363] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:06 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:26:06.986139 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6452f2-17dc-484a-bcca-443d2040aa2c admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:07.013519 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6452f2-17dc-484a-bcca-443d2040aa2c admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:07.025321 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:07.079688 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f8e19-f87a-4a8e-a820-07b3168dbe4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:07.080470 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f8e19-f87a-4a8e-a820-07b3168dbe4d admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:07.080756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f8e19-f87a-4a8e-a820-07b3168dbe4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:07.081026 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f8e19-f87a-4a8e-a820-07b3168dbe4d admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:07.081684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f8e19-f87a-4a8e-a820-07b3168dbe4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:07.084322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f8e19-f87a-4a8e-a820-07b3168dbe4d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:07.111722 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1173/2364] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:06 2022] POST /identity/v3/users => generated 286 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:07.112950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c794f9-c4d5-431c-bcd5-3a1f88131c78 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:07.125354 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c794f9-c4d5-431c-bcd5-3a1f88131c78 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:07.132227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc2bea-0ab9-4ca4-9d83-c099ade17122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:07.133277 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc2bea-0ab9-4ca4-9d83-c099ade17122 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:07.133803 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc2bea-0ab9-4ca4-9d83-c099ade17122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:07.134171 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc2bea-0ab9-4ca4-9d83-c099ade17122 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:07.134964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dc2bea-0ab9-4ca4-9d83-c099ade17122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:07.137627 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dc2bea-0ab9-4ca4-9d83-c099ade17122 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:07.148848 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1192/2365] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:07.152805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f8e19-f87a-4a8e-a820-07b3168dbe4d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:07.163649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f8e19-f87a-4a8e-a820-07b3168dbe4d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:07.168468 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e18b62-12a4-443a-8d8c-2c100f640be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:07.169063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e18b62-12a4-443a-8d8c-2c100f640be7 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:07.169345 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e18b62-12a4-443a-8d8c-2c100f640be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:07.169618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e18b62-12a4-443a-8d8c-2c100f640be7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:07.170251 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e18b62-12a4-443a-8d8c-2c100f640be7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:07.171876 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e18b62-12a4-443a-8d8c-2c100f640be7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:07.183167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1174/2366] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:07.187172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc2bea-0ab9-4ca4-9d83-c099ade17122 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:07.198256 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc2bea-0ab9-4ca4-9d83-c099ade17122 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:07.203092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b3b6d-02bc-4e87-bf7e-6bf2a1b0f957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:07.203738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b3b6d-02bc-4e87-bf7e-6bf2a1b0f957 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:07.204047 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b3b6d-02bc-4e87-bf7e-6bf2a1b0f957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:07.204318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b3b6d-02bc-4e87-bf7e-6bf2a1b0f957 admin admin] PATH_INFO: `/v3/projects/e14cb95e59a34903a938b53ea2757059/users/1c493883d4e743209322fc8b897e14fa/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:07.221495 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b3b6d-02bc-4e87-bf7e-6bf2a1b0f957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e14cb95e59a34903a938b53ea2757059, user_id=1c493883d4e743209322fc8b897e14fa, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:07.224178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b3b6d-02bc-4e87-bf7e-6bf2a1b0f957 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:07.238527 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1193/2367] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:07 2022] PUT /identity/v3/projects/e14cb95e59a34903a938b53ea2757059/users/1c493883d4e743209322fc8b897e14fa/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:26:07.243415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb33ca-5e57-4be3-94ea-76aa7c94ddb5 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:07.243705 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb33ca-5e57-4be3-94ea-76aa7c94ddb5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:07.244101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb33ca-5e57-4be3-94ea-76aa7c94ddb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:07.263469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e0bb33ca-5e57-4be3-94ea-76aa7c94ddb5 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:07.264736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e0bb33ca-5e57-4be3-94ea-76aa7c94ddb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:07.325600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1175/2368] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:07.336194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b3b6d-02bc-4e87-bf7e-6bf2a1b0f957 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:07.338002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b04b3b6d-02bc-4e87-bf7e-6bf2a1b0f957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:07.394784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b3b6d-02bc-4e87-bf7e-6bf2a1b0f957 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:07.398971 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6f8e4-b7d9-4f37-9798-3e3fc84552f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:07.399564 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6f8e4-b7d9-4f37-9798-3e3fc84552f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:07.399876 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6f8e4-b7d9-4f37-9798-3e3fc84552f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:07.400151 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6f8e4-b7d9-4f37-9798-3e3fc84552f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:07.408968 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6f8e4-b7d9-4f37-9798-3e3fc84552f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:07.410372 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6f8e4-b7d9-4f37-9798-3e3fc84552f5 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:07.420556 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1194/2369] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:08.203659 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e45fc-d551-40cf-8e16-9bcedf577b3e None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:08.204223 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e45fc-d551-40cf-8e16-9bcedf577b3e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:08.204338 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e45fc-d551-40cf-8e16-9bcedf577b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:08.233528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-4c8e45fc-d551-40cf-8e16-9bcedf577b3e None None] MFA Rules not processed for user `1c493883d4e743209322fc8b897e14fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:08.234767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4c8e45fc-d551-40cf-8e16-9bcedf577b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:08.312003 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1176/2370] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:26:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:08.319287 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6f8e4-b7d9-4f37-9798-3e3fc84552f5 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:08.329852 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6f8e4-b7d9-4f37-9798-3e3fc84552f5 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:08.335412 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79ed79-9234-4e4b-ace1-568e4e6c2c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:08.336135 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79ed79-9234-4e4b-ace1-568e4e6c2c16 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:08.336517 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79ed79-9234-4e4b-ace1-568e4e6c2c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:08.336889 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79ed79-9234-4e4b-ace1-568e4e6c2c16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:08.337669 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79ed79-9234-4e4b-ace1-568e4e6c2c16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:08.339850 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79ed79-9234-4e4b-ace1-568e4e6c2c16 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:08.375643 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1195/2371] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:08 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:26:08.376771 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e45fc-d551-40cf-8e16-9bcedf577b3e None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:08.392324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e45fc-d551-40cf-8e16-9bcedf577b3e None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:08.411190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:08.457024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a331fb9-2730-4945-92a7-25621bb52693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:08.457895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a331fb9-2730-4945-92a7-25621bb52693 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:08.458173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a331fb9-2730-4945-92a7-25621bb52693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:08.458445 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a331fb9-2730-4945-92a7-25621bb52693 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:08.459643 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a331fb9-2730-4945-92a7-25621bb52693 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:08.461620 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a331fb9-2730-4945-92a7-25621bb52693 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:08.489516 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1177/2372] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:08 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:26:08.491018 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79ed79-9234-4e4b-ace1-568e4e6c2c16 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:08.501821 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79ed79-9234-4e4b-ace1-568e4e6c2c16 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:08.506996 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af49336-9296-49d6-8162-e1d4b809f18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:08.507601 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af49336-9296-49d6-8162-e1d4b809f18b admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:08.507909 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af49336-9296-49d6-8162-e1d4b809f18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:08.508184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af49336-9296-49d6-8162-e1d4b809f18b admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:08.509136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af49336-9296-49d6-8162-e1d4b809f18b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:08.510774 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af49336-9296-49d6-8162-e1d4b809f18b admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:08.522613 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1196/2373] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:08.526445 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a331fb9-2730-4945-92a7-25621bb52693 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:08.536481 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a331fb9-2730-4945-92a7-25621bb52693 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:08.541223 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a6896-d625-414d-b7a5-76f76be08853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:08.541863 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a6896-d625-414d-b7a5-76f76be08853 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:08.542145 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a6896-d625-414d-b7a5-76f76be08853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:08.542418 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a6896-d625-414d-b7a5-76f76be08853 admin admin] PATH_INFO: `/v3/projects/5e9c83c2530240c0903bdbbc335c2e60/users/9ba7725812294e5b8cca6ea5228a6f1b/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:08.557111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a6896-d625-414d-b7a5-76f76be08853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e9c83c2530240c0903bdbbc335c2e60, user_id=9ba7725812294e5b8cca6ea5228a6f1b, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:08.559731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a6896-d625-414d-b7a5-76f76be08853 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:08.573978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1178/2374] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:08 2022] PUT /identity/v3/projects/5e9c83c2530240c0903bdbbc335c2e60/users/9ba7725812294e5b8cca6ea5228a6f1b/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:08.578168 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af49336-9296-49d6-8162-e1d4b809f18b admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:08.589075 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af49336-9296-49d6-8162-e1d4b809f18b admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:08.628968 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:08.650982 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263d59e-6ff7-4d67-ab16-d301baaabaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:08.651601 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263d59e-6ff7-4d67-ab16-d301baaabaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:08.651920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263d59e-6ff7-4d67-ab16-d301baaabaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:08.652195 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263d59e-6ff7-4d67-ab16-d301baaabaa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:08.652840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263d59e-6ff7-4d67-ab16-d301baaabaa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:08.654426 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263d59e-6ff7-4d67-ab16-d301baaabaa1 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:08.668150 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1197/2375] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:08.669318 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a6896-d625-414d-b7a5-76f76be08853 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:08.680412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a6896-d625-414d-b7a5-76f76be08853 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:08.686724 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953dde3d-9dd0-4296-a15c-9e0609aa6f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:08.687419 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953dde3d-9dd0-4296-a15c-9e0609aa6f0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:08.687701 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953dde3d-9dd0-4296-a15c-9e0609aa6f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:08.688005 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953dde3d-9dd0-4296-a15c-9e0609aa6f0e admin admin] PATH_INFO: `/v3/projects/5e9c83c2530240c0903bdbbc335c2e60/users/9ba7725812294e5b8cca6ea5228a6f1b/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:08.689972 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953dde3d-9dd0-4296-a15c-9e0609aa6f0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e9c83c2530240c0903bdbbc335c2e60, user_id=9ba7725812294e5b8cca6ea5228a6f1b, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:08.692649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953dde3d-9dd0-4296-a15c-9e0609aa6f0e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:08.704146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1179/2376] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:08 2022] PUT /identity/v3/projects/5e9c83c2530240c0903bdbbc335c2e60/users/9ba7725812294e5b8cca6ea5228a6f1b/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:09.441928 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bbe05-29cd-4fe4-8629-04f95f0b89ae None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:09.443295 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bbe05-29cd-4fe4-8629-04f95f0b89ae None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:09.443798 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bbe05-29cd-4fe4-8629-04f95f0b89ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:09.487811 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-8f2bbe05-29cd-4fe4-8629-04f95f0b89ae None None] MFA Rules not processed for user `9ba7725812294e5b8cca6ea5228a6f1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:09.489032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-8f2bbe05-29cd-4fe4-8629-04f95f0b89ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:09.587519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1198/2377] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:26:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:09.600562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953dde3d-9dd0-4296-a15c-9e0609aa6f0e admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:09.602367 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-953dde3d-9dd0-4296-a15c-9e0609aa6f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:09.692837 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953dde3d-9dd0-4296-a15c-9e0609aa6f0e admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:09.697200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60a03c-b686-4f53-9f24-457dc57615f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:09.697817 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60a03c-b686-4f53-9f24-457dc57615f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:09.698227 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60a03c-b686-4f53-9f24-457dc57615f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:09.698642 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60a03c-b686-4f53-9f24-457dc57615f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:09.806155 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60a03c-b686-4f53-9f24-457dc57615f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:09.808201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60a03c-b686-4f53-9f24-457dc57615f1 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:09.824181 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1180/2378] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:11.060487 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ac611-5e11-486b-984a-ec2f217349e9 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:11.061412 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ac611-5e11-486b-984a-ec2f217349e9 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:11.061913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ac611-5e11-486b-984a-ec2f217349e9 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:11.064052 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1199/2379] 10.222.0.81 () {54 vars in 921 bytes} [Mon Sep 19 11:26:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:26:11.078527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c43b2b-e8b7-49c4-a37d-3721553bb0fe None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:11.079533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c43b2b-e8b7-49c4-a37d-3721553bb0fe None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:11.080094 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c43b2b-e8b7-49c4-a37d-3721553bb0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:11.108258 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d8c43b2b-e8b7-49c4-a37d-3721553bb0fe None None] MFA Rules not processed for user `5abd70a18ccd407bafe3b61c8870986e`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:11.109851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d8c43b2b-e8b7-49c4-a37d-3721553bb0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:11.137522 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:11.194137 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1181/2380] 10.222.0.81 () {58 vars in 1018 bytes} [Mon Sep 19 11:26:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:11.206245 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ac611-5e11-486b-984a-ec2f217349e9 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:11.208492 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-643ac611-5e11-486b-984a-ec2f217349e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:11.273220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ac611-5e11-486b-984a-ec2f217349e9 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:11.279310 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f5b315-dab2-484c-9b9a-7194030b2f0a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88670b0e1a0f460d8f74917c92db9b9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:11.280005 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5b315-dab2-484c-9b9a-7194030b2f0a service glance] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:11.280230 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5b315-dab2-484c-9b9a-7194030b2f0a service glance] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:11.280506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5b315-dab2-484c-9b9a-7194030b2f0a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:11.305906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f5b315-dab2-484c-9b9a-7194030b2f0a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:11.307559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f5b315-dab2-484c-9b9a-7194030b2f0a service glance] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:11.332025 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1200/2381] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:11.356280 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c43b2b-e8b7-49c4-a37d-3721553bb0fe None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:11.357720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d8c43b2b-e8b7-49c4-a37d-3721553bb0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:11.370873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c43b2b-e8b7-49c4-a37d-3721553bb0fe None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:11.374874 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b8680-0e0c-46ff-8158-b20a9f62fc97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:11.375504 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b8680-0e0c-46ff-8158-b20a9f62fc97 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:11.375807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b8680-0e0c-46ff-8158-b20a9f62fc97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:11.376258 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b8680-0e0c-46ff-8158-b20a9f62fc97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:11.386973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b8680-0e0c-46ff-8158-b20a9f62fc97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:11.388495 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b8680-0e0c-46ff-8158-b20a9f62fc97 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:11.399806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1182/2382] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:11.424993 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da6ac7-0145-44ce-aba4-69467a2821b7 None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:11.425424 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da6ac7-0145-44ce-aba4-69467a2821b7 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:11.425576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da6ac7-0145-44ce-aba4-69467a2821b7 None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:11.427053 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1201/2383] 10.222.0.81 () {54 vars in 921 bytes} [Mon Sep 19 11:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:26:11.431964 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcef1f7-5e50-4fcf-9217-546644dfa067 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:11.432257 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcef1f7-5e50-4fcf-9217-546644dfa067 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:11.432536 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcef1f7-5e50-4fcf-9217-546644dfa067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:11.460563 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-ffcef1f7-5e50-4fcf-9217-546644dfa067 None None] MFA Rules not processed for user `65de2e5f67dd4bb6a20cf3934afc8abd`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:11.465569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ffcef1f7-5e50-4fcf-9217-546644dfa067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:11.539909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1183/2384] 10.222.0.81 () {58 vars in 1018 bytes} [Mon Sep 19 11:26:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:11.552654 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da6ac7-0145-44ce-aba4-69467a2821b7 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:11.554434 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-34da6ac7-0145-44ce-aba4-69467a2821b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:11.572471 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da6ac7-0145-44ce-aba4-69467a2821b7 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:11.576209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc3c30-a2b8-4a60-8051-b3a03412b553 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:11.576802 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc3c30-a2b8-4a60-8051-b3a03412b553 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:11.577080 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc3c30-a2b8-4a60-8051-b3a03412b553 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:11.577353 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc3c30-a2b8-4a60-8051-b3a03412b553 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:11.591227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc3c30-a2b8-4a60-8051-b3a03412b553 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:11.593019 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc3c30-a2b8-4a60-8051-b3a03412b553 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:11.603716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1202/2385] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:11.743991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3713f90-8f53-468d-9fd4-a470ee0ee583 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:11.744442 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3713f90-8f53-468d-9fd4-a470ee0ee583 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:11.744784 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3713f90-8f53-468d-9fd4-a470ee0ee583 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:11.747497 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1184/2386] 10.222.0.81 () {54 vars in 921 bytes} [Mon Sep 19 11:26:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:26:11.761135 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89907524-65d6-4a99-b280-72a1cf88c284 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:11.761788 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89907524-65d6-4a99-b280-72a1cf88c284 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:11.762190 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89907524-65d6-4a99-b280-72a1cf88c284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:11.783451 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-89907524-65d6-4a99-b280-72a1cf88c284 None None] MFA Rules not processed for user `5abd70a18ccd407bafe3b61c8870986e`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:11.784993 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-89907524-65d6-4a99-b280-72a1cf88c284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:11.797479 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1203/2387] 10.222.0.81 () {58 vars in 1018 bytes} [Mon Sep 19 11:26:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:11.810629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3713f90-8f53-468d-9fd4-a470ee0ee583 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:11.834439 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3713f90-8f53-468d-9fd4-a470ee0ee583 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:11.844789 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8738e-a63f-40ba-a9db-963041812c95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:11.845392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8738e-a63f-40ba-a9db-963041812c95 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:11.846374 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8738e-a63f-40ba-a9db-963041812c95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:11.846693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8738e-a63f-40ba-a9db-963041812c95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:11.863272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8738e-a63f-40ba-a9db-963041812c95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:11.866653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8738e-a63f-40ba-a9db-963041812c95 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:11.894581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1185/2388] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:15.195002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f429c2d-4f8a-484d-a4e3-48a751781570 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:15.195498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f429c2d-4f8a-484d-a4e3-48a751781570 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:15.195609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f429c2d-4f8a-484d-a4e3-48a751781570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:15.215268 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-8f429c2d-4f8a-484d-a4e3-48a751781570 None None] MFA Rules not processed for user `06d279b488d148428b7a3c35a9786ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:15.216499 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-8f429c2d-4f8a-484d-a4e3-48a751781570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:15.257086 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:15.270714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1204/2389] 10.222.0.81 () {58 vars in 1009 bytes} [Mon Sep 19 11:26:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:15.282869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8738e-a63f-40ba-a9db-963041812c95 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:15.284283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-53d8738e-a63f-40ba-a9db-963041812c95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:15.356468 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8738e-a63f-40ba-a9db-963041812c95 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:15.361434 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84031dec-8e49-4aba-9bcc-6ab198120a83 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8a9732807d4921a41a6a2d5019e2f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:15.362040 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84031dec-8e49-4aba-9bcc-6ab198120a83 service swift] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:15.362436 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84031dec-8e49-4aba-9bcc-6ab198120a83 service swift] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:15.362715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84031dec-8e49-4aba-9bcc-6ab198120a83 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:15.372591 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84031dec-8e49-4aba-9bcc-6ab198120a83 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:15.374452 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84031dec-8e49-4aba-9bcc-6ab198120a83 service swift] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:15.382919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1186/2390] 10.222.0.81 () {60 vars in 1384 bytes} [Mon Sep 19 11:26:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 11:26:25.105925 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0c223-8491-4051-9f12-f140e7acc8ee None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:25.106428 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0c223-8491-4051-9f12-f140e7acc8ee None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:25.106606 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0c223-8491-4051-9f12-f140e7acc8ee None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:25.108465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1205/2391] 10.222.0.81 () {54 vars in 921 bytes} [Mon Sep 19 11:26:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:26:25.118568 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e221c82-3646-4df2-a34c-e3c83c93c27b None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:25.119125 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e221c82-3646-4df2-a34c-e3c83c93c27b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:25.119125 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e221c82-3646-4df2-a34c-e3c83c93c27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:25.147696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9e221c82-3646-4df2-a34c-e3c83c93c27b None None] MFA Rules not processed for user `6e7a18dc4a4d4d35a05f1aafbaaf9c31`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:25.148956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9e221c82-3646-4df2-a34c-e3c83c93c27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:25.175619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:25.215157 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1187/2392] 10.222.0.81 () {58 vars in 1018 bytes} [Mon Sep 19 11:26:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:25.230173 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0c223-8491-4051-9f12-f140e7acc8ee None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:25.236074 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e221c82-3646-4df2-a34c-e3c83c93c27b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:25.237908 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9e221c82-3646-4df2-a34c-e3c83c93c27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:25.251938 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e221c82-3646-4df2-a34c-e3c83c93c27b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:25.262890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb79a5-5c83-4e81-84d3-0ee11a443e04 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e7a18dc4a4d4d35a05f1aafbaaf9c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:25.263936 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb79a5-5c83-4e81-84d3-0ee11a443e04 service placement] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:25.264574 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb79a5-5c83-4e81-84d3-0ee11a443e04 service placement] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:25.265021 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb79a5-5c83-4e81-84d3-0ee11a443e04 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:25.278781 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb79a5-5c83-4e81-84d3-0ee11a443e04 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:25.286283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb79a5-5c83-4e81-84d3-0ee11a443e04 service placement] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:25.299157 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1188/2393] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:26:25.320895 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0c223-8491-4051-9f12-f140e7acc8ee None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:25.325707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff1e24-67ca-4c3d-9cae-c265328a5381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:25.326333 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff1e24-67ca-4c3d-9cae-c265328a5381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:25.326610 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff1e24-67ca-4c3d-9cae-c265328a5381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:25.326879 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff1e24-67ca-4c3d-9cae-c265328a5381 admin admin] PATH_INFO: `/v3/users/1c493883d4e743209322fc8b897e14fa` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:25.328048 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fff1e24-67ca-4c3d-9cae-c265328a5381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c493883d4e743209322fc8b897e14fa)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:25.329973 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fff1e24-67ca-4c3d-9cae-c265328a5381 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:25.378485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-9fff1e24-67ca-4c3d-9cae-c265328a5381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c493883d4e743209322fc8b897e14fa'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:25.379146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-9fff1e24-67ca-4c3d-9cae-c265328a5381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c493883d4e743209322fc8b897e14fa'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:25.379776 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-9fff1e24-67ca-4c3d-9cae-c265328a5381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c493883d4e743209322fc8b897e14fa'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:25.386560 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-9fff1e24-67ca-4c3d-9cae-c265328a5381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c493883d4e743209322fc8b897e14fa'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:25.398547 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-9fff1e24-67ca-4c3d-9cae-c265328a5381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c493883d4e743209322fc8b897e14fa'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:25.410824 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1206/2394] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:25 2022] DELETE /identity/v3/users/1c493883d4e743209322fc8b897e14fa => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:25.584094 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb79a5-5c83-4e81-84d3-0ee11a443e04 service placement] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:25.594794 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb79a5-5c83-4e81-84d3-0ee11a443e04 service placement] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:25.660061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babbd77-b939-4e6e-aa02-3286e9b0bfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:25.660704 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babbd77-b939-4e6e-aa02-3286e9b0bfd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:25.660985 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babbd77-b939-4e6e-aa02-3286e9b0bfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:25.661749 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babbd77-b939-4e6e-aa02-3286e9b0bfd9 admin admin] PATH_INFO: `/v3/projects/e14cb95e59a34903a938b53ea2757059` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:25.662887 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1babbd77-b939-4e6e-aa02-3286e9b0bfd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e14cb95e59a34903a938b53ea2757059)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:25.664873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1babbd77-b939-4e6e-aa02-3286e9b0bfd9 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:25.712551 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1babbd77-b939-4e6e-aa02-3286e9b0bfd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e14cb95e59a34903a938b53ea2757059', 'request_id': 'req-1babbd77-b939-4e6e-aa02-3286e9b0bfd9'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:25.718901 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1babbd77-b939-4e6e-aa02-3286e9b0bfd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e14cb95e59a34903a938b53ea2757059', 'request_id': 'req-1babbd77-b939-4e6e-aa02-3286e9b0bfd9'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:25.727474 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1babbd77-b939-4e6e-aa02-3286e9b0bfd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e14cb95e59a34903a938b53ea2757059', 'request_id': 'req-1babbd77-b939-4e6e-aa02-3286e9b0bfd9'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:25.741192 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1babbd77-b939-4e6e-aa02-3286e9b0bfd9 admin admin] The token cache is being invalidate because project e14cb95e59a34903a938b53ea2757059 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:25.741485 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1babbd77-b939-4e6e-aa02-3286e9b0bfd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:25.742144 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-1babbd77-b939-4e6e-aa02-3286e9b0bfd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:25.743630 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1189/2395] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:25 2022] DELETE /identity/v3/projects/e14cb95e59a34903a938b53ea2757059 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:25.747723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff1e24-67ca-4c3d-9cae-c265328a5381 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:25.749307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-9fff1e24-67ca-4c3d-9cae-c265328a5381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:25.762858 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff1e24-67ca-4c3d-9cae-c265328a5381 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:25.828332 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e2e7e-c4f9-4b0a-9a88-c11cb6350b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:25.829108 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e2e7e-c4f9-4b0a-9a88-c11cb6350b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:25.829494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e2e7e-c4f9-4b0a-9a88-c11cb6350b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:25.829873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e2e7e-c4f9-4b0a-9a88-c11cb6350b4f admin admin] PATH_INFO: `/v3/users/9ba7725812294e5b8cca6ea5228a6f1b` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:25.831158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e2e7e-c4f9-4b0a-9a88-c11cb6350b4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ba7725812294e5b8cca6ea5228a6f1b)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:25.834154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e2e7e-c4f9-4b0a-9a88-c11cb6350b4f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:25.847259 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:25.892310 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e11e2e7e-c4f9-4b0a-9a88-c11cb6350b4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ba7725812294e5b8cca6ea5228a6f1b'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:25.893321 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e11e2e7e-c4f9-4b0a-9a88-c11cb6350b4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ba7725812294e5b8cca6ea5228a6f1b'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:25.893607 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e11e2e7e-c4f9-4b0a-9a88-c11cb6350b4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ba7725812294e5b8cca6ea5228a6f1b'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:25.901226 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e11e2e7e-c4f9-4b0a-9a88-c11cb6350b4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ba7725812294e5b8cca6ea5228a6f1b'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:25.914171 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e11e2e7e-c4f9-4b0a-9a88-c11cb6350b4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ba7725812294e5b8cca6ea5228a6f1b'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:25.928283 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1207/2396] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:25 2022] DELETE /identity/v3/users/9ba7725812294e5b8cca6ea5228a6f1b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:26.105116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babbd77-b939-4e6e-aa02-3286e9b0bfd9 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:26.106537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1babbd77-b939-4e6e-aa02-3286e9b0bfd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:26.120753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babbd77-b939-4e6e-aa02-3286e9b0bfd9 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:26.166290 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:26.183445 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b72543-d6de-469e-be22-f24141f44b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:26.184122 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b72543-d6de-469e-be22-f24141f44b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:26.184429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b72543-d6de-469e-be22-f24141f44b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:26.184705 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b72543-d6de-469e-be22-f24141f44b23 admin admin] PATH_INFO: `/v3/projects/5e9c83c2530240c0903bdbbc335c2e60` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:26.185829 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b72543-d6de-469e-be22-f24141f44b23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e9c83c2530240c0903bdbbc335c2e60)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:26.187810 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b72543-d6de-469e-be22-f24141f44b23 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:26.235322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-75b72543-d6de-469e-be22-f24141f44b23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e9c83c2530240c0903bdbbc335c2e60', 'request_id': 'req-75b72543-d6de-469e-be22-f24141f44b23'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:26.242633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-75b72543-d6de-469e-be22-f24141f44b23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e9c83c2530240c0903bdbbc335c2e60', 'request_id': 'req-75b72543-d6de-469e-be22-f24141f44b23'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:26.248396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-75b72543-d6de-469e-be22-f24141f44b23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e9c83c2530240c0903bdbbc335c2e60', 'request_id': 'req-75b72543-d6de-469e-be22-f24141f44b23'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:26.262038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-75b72543-d6de-469e-be22-f24141f44b23 admin admin] The token cache is being invalidate because project 5e9c83c2530240c0903bdbbc335c2e60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:26.262335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-75b72543-d6de-469e-be22-f24141f44b23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:26.262968 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-75b72543-d6de-469e-be22-f24141f44b23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:26.264528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1190/2397] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:26 2022] DELETE /identity/v3/projects/5e9c83c2530240c0903bdbbc335c2e60 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:26.270717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bcc18-3e7e-402e-8d4e-8efada0c4861 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:26.271250 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bcc18-3e7e-402e-8d4e-8efada0c4861 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:26.271357 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bcc18-3e7e-402e-8d4e-8efada0c4861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:26.291863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-bf3bcc18-3e7e-402e-8d4e-8efada0c4861 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:26.293091 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-bf3bcc18-3e7e-402e-8d4e-8efada0c4861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:26.359647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1208/2398] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:26.367798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d08d5-bec2-4c4a-aa56-2d4e413113ef None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:26.368362 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d08d5-bec2-4c4a-aa56-2d4e413113ef None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:26.368646 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d08d5-bec2-4c4a-aa56-2d4e413113ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:26.389262 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-671d08d5-bec2-4c4a-aa56-2d4e413113ef None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:26.390490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-671d08d5-bec2-4c4a-aa56-2d4e413113ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:26.398488 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1191/2399] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:26.402766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bcc18-3e7e-402e-8d4e-8efada0c4861 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:26.413800 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bcc18-3e7e-402e-8d4e-8efada0c4861 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:26.418607 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992c7b9-39e5-4f68-8dde-a21e035ad0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:26.419224 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992c7b9-39e5-4f68-8dde-a21e035ad0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:26.419635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992c7b9-39e5-4f68-8dde-a21e035ad0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:26.419841 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992c7b9-39e5-4f68-8dde-a21e035ad0e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:26.420527 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992c7b9-39e5-4f68-8dde-a21e035ad0e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:26:26.420830 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992c7b9-39e5-4f68-8dde-a21e035ad0e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:26.422459 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992c7b9-39e5-4f68-8dde-a21e035ad0e6 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:26.436081 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1209/2400] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:26:26.440574 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d08d5-bec2-4c4a-aa56-2d4e413113ef None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:26.450032 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d08d5-bec2-4c4a-aa56-2d4e413113ef None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:26.454740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb2ed0-8663-403d-a2c2-2c39f17f40f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:26.455341 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb2ed0-8663-403d-a2c2-2c39f17f40f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:26.455622 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb2ed0-8663-403d-a2c2-2c39f17f40f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:26.455927 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb2ed0-8663-403d-a2c2-2c39f17f40f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:26.456561 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb2ed0-8663-403d-a2c2-2c39f17f40f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:26.458763 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb2ed0-8663-403d-a2c2-2c39f17f40f7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:26.491613 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1192/2401] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:26 2022] POST /identity/v3/projects => generated 397 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:26:26.495741 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992c7b9-39e5-4f68-8dde-a21e035ad0e6 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:26.505938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992c7b9-39e5-4f68-8dde-a21e035ad0e6 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:26.562170 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c113ef-569f-4b70-97d3-927c198e8634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:26.562818 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c113ef-569f-4b70-97d3-927c198e8634 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:26.563152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c113ef-569f-4b70-97d3-927c198e8634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:26.563421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c113ef-569f-4b70-97d3-927c198e8634 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:26.564138 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c113ef-569f-4b70-97d3-927c198e8634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:26.566080 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c113ef-569f-4b70-97d3-927c198e8634 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:26.588039 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1210/2402] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:26 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:26:26.593124 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb2ed0-8663-403d-a2c2-2c39f17f40f7 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:26.604305 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb2ed0-8663-403d-a2c2-2c39f17f40f7 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:26.608967 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26ab31-d8b1-4864-9ed2-64fbf648687b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:26.609565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26ab31-d8b1-4864-9ed2-64fbf648687b admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:26.609900 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26ab31-d8b1-4864-9ed2-64fbf648687b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:26.610113 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26ab31-d8b1-4864-9ed2-64fbf648687b admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:26.610754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26ab31-d8b1-4864-9ed2-64fbf648687b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:26.612439 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26ab31-d8b1-4864-9ed2-64fbf648687b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:26.622855 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1193/2403] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:26.626730 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c113ef-569f-4b70-97d3-927c198e8634 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:26.636791 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c113ef-569f-4b70-97d3-927c198e8634 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:26.641420 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61e476-2e8a-45df-a414-74b5265e0647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:26.642007 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61e476-2e8a-45df-a414-74b5265e0647 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:26.642286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61e476-2e8a-45df-a414-74b5265e0647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:26.642559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61e476-2e8a-45df-a414-74b5265e0647 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:26.643196 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61e476-2e8a-45df-a414-74b5265e0647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:26.644799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61e476-2e8a-45df-a414-74b5265e0647 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:26.646545 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:26.657647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1211/2404] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:26.661436 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26ab31-d8b1-4864-9ed2-64fbf648687b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:26.672247 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26ab31-d8b1-4864-9ed2-64fbf648687b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:26.677420 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a41967-fdbf-4974-a73a-3c86aaf431d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:26.678091 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a41967-fdbf-4974-a73a-3c86aaf431d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:26.678398 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a41967-fdbf-4974-a73a-3c86aaf431d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:26.678677 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a41967-fdbf-4974-a73a-3c86aaf431d3 admin admin] PATH_INFO: `/v3/projects/0e84f6dfc7db4dbc8a0f452e10cf4dec/users/d6f6009dfa4f407f855b7400048a0844/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:26.693834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a41967-fdbf-4974-a73a-3c86aaf431d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e84f6dfc7db4dbc8a0f452e10cf4dec, user_id=d6f6009dfa4f407f855b7400048a0844, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:26.696571 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a41967-fdbf-4974-a73a-3c86aaf431d3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:26.709286 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1194/2405] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:26 2022] PUT /identity/v3/projects/0e84f6dfc7db4dbc8a0f452e10cf4dec/users/d6f6009dfa4f407f855b7400048a0844/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:26.719636 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a8eca-1db7-42d1-ab9a-722679dd0b91 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:26.719961 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a8eca-1db7-42d1-ab9a-722679dd0b91 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:26.720241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a8eca-1db7-42d1-ab9a-722679dd0b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:26.753578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-010a8eca-1db7-42d1-ab9a-722679dd0b91 None None] MFA Rules not processed for user `d6f6009dfa4f407f855b7400048a0844`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:26.754801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-010a8eca-1db7-42d1-ab9a-722679dd0b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:26.817738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1212/2406] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:26:26 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:26.829163 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a41967-fdbf-4974-a73a-3c86aaf431d3 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:26.832328 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-70a41967-fdbf-4974-a73a-3c86aaf431d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:26.891248 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a41967-fdbf-4974-a73a-3c86aaf431d3 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:26.895040 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21ace8-cd88-4485-a4e8-7556689f75d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:26.895855 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21ace8-cd88-4485-a4e8-7556689f75d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:26.896112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21ace8-cd88-4485-a4e8-7556689f75d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:26.896416 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21ace8-cd88-4485-a4e8-7556689f75d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:26.905086 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af21ace8-cd88-4485-a4e8-7556689f75d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:26.906523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af21ace8-cd88-4485-a4e8-7556689f75d6 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:26.915790 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1195/2407] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:26 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:34.448321 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a8eca-1db7-42d1-ab9a-722679dd0b91 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:34.449804 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-010a8eca-1db7-42d1-ab9a-722679dd0b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:34.478670 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21ace8-cd88-4485-a4e8-7556689f75d6 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:34.480168 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-af21ace8-cd88-4485-a4e8-7556689f75d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:34.494903 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:34.509883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a8eca-1db7-42d1-ab9a-722679dd0b91 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:34.513988 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9813f16-def9-427b-8f0b-e3e577ff3ae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:34.514625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9813f16-def9-427b-8f0b-e3e577ff3ae0 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:34.514982 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9813f16-def9-427b-8f0b-e3e577ff3ae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:34.515245 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9813f16-def9-427b-8f0b-e3e577ff3ae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:34.517094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e9813f16-def9-427b-8f0b-e3e577ff3ae0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:34.557371 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21ace8-cd88-4485-a4e8-7556689f75d6 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:34.562619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38177900-e881-4ad5-8049-35b389173128 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e7a18dc4a4d4d35a05f1aafbaaf9c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:34.563231 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38177900-e881-4ad5-8049-35b389173128 service placement] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:34.563520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38177900-e881-4ad5-8049-35b389173128 service placement] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:34.563815 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38177900-e881-4ad5-8049-35b389173128 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:34.565629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-38177900-e881-4ad5-8049-35b389173128 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:34.577924 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38177900-e881-4ad5-8049-35b389173128 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:34.579523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38177900-e881-4ad5-8049-35b389173128 service placement] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:34.582114 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9813f16-def9-427b-8f0b-e3e577ff3ae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:34.583790 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9813f16-def9-427b-8f0b-e3e577ff3ae0 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:34.589893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1196/2408] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:26:34.593596 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1213/2409] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:35.015689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38177900-e881-4ad5-8049-35b389173128 service placement] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:35.082228 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38177900-e881-4ad5-8049-35b389173128 service placement] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:35.087579 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065de48-57df-4cdc-9d4a-e276285750aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:35.088239 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065de48-57df-4cdc-9d4a-e276285750aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.088512 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065de48-57df-4cdc-9d4a-e276285750aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.088780 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065de48-57df-4cdc-9d4a-e276285750aa admin admin] PATH_INFO: `/v3/users/d6f6009dfa4f407f855b7400048a0844` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.089896 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065de48-57df-4cdc-9d4a-e276285750aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6f6009dfa4f407f855b7400048a0844)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:35.091835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065de48-57df-4cdc-9d4a-e276285750aa admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:35.144329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b065de48-57df-4cdc-9d4a-e276285750aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6f6009dfa4f407f855b7400048a0844'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:35.152105 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b065de48-57df-4cdc-9d4a-e276285750aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6f6009dfa4f407f855b7400048a0844'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:35.152712 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b065de48-57df-4cdc-9d4a-e276285750aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6f6009dfa4f407f855b7400048a0844'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:35.153341 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b065de48-57df-4cdc-9d4a-e276285750aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6f6009dfa4f407f855b7400048a0844'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:35.166205 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b065de48-57df-4cdc-9d4a-e276285750aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6f6009dfa4f407f855b7400048a0844'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:35.182361 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1197/2410] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:26:35 2022] DELETE /identity/v3/users/d6f6009dfa4f407f855b7400048a0844 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:35.184097 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea808e-2f58-4c6d-a0e7-f1ed82cc2756 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.184383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea808e-2f58-4c6d-a0e7-f1ed82cc2756 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.184672 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea808e-2f58-4c6d-a0e7-f1ed82cc2756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.204105 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-8fea808e-2f58-4c6d-a0e7-f1ed82cc2756 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:35.205310 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-8fea808e-2f58-4c6d-a0e7-f1ed82cc2756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:35.237307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:35.266506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1214/2411] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:35.276654 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065de48-57df-4cdc-9d4a-e276285750aa admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:35.278171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b065de48-57df-4cdc-9d4a-e276285750aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:35.305959 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:35.344949 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065de48-57df-4cdc-9d4a-e276285750aa admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:35.348946 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ca0e7-20ac-4ca4-9604-e38918f144a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:35.349686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ca0e7-20ac-4ca4-9604-e38918f144a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.350000 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ca0e7-20ac-4ca4-9604-e38918f144a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.350341 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ca0e7-20ac-4ca4-9604-e38918f144a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.359635 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ca0e7-20ac-4ca4-9604-e38918f144a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:35.361098 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ca0e7-20ac-4ca4-9604-e38918f144a3 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:35.371420 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1198/2412] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:35.657755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea808e-2f58-4c6d-a0e7-f1ed82cc2756 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:35.660218 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-8fea808e-2f58-4c6d-a0e7-f1ed82cc2756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:35.686678 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea808e-2f58-4c6d-a0e7-f1ed82cc2756 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:35.692838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25eb3bc-cfdf-410d-b15c-b2acb9adf5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:35.693475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25eb3bc-cfdf-410d-b15c-b2acb9adf5b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.693759 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25eb3bc-cfdf-410d-b15c-b2acb9adf5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.694062 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25eb3bc-cfdf-410d-b15c-b2acb9adf5b8 admin admin] PATH_INFO: `/v3/projects/0e84f6dfc7db4dbc8a0f452e10cf4dec` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.695228 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25eb3bc-cfdf-410d-b15c-b2acb9adf5b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e84f6dfc7db4dbc8a0f452e10cf4dec)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:35.697412 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25eb3bc-cfdf-410d-b15c-b2acb9adf5b8 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:35.751632 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b25eb3bc-cfdf-410d-b15c-b2acb9adf5b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e84f6dfc7db4dbc8a0f452e10cf4dec', 'request_id': 'req-b25eb3bc-cfdf-410d-b15c-b2acb9adf5b8'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:35.759370 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b25eb3bc-cfdf-410d-b15c-b2acb9adf5b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e84f6dfc7db4dbc8a0f452e10cf4dec', 'request_id': 'req-b25eb3bc-cfdf-410d-b15c-b2acb9adf5b8'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:35.766505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b25eb3bc-cfdf-410d-b15c-b2acb9adf5b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e84f6dfc7db4dbc8a0f452e10cf4dec', 'request_id': 'req-b25eb3bc-cfdf-410d-b15c-b2acb9adf5b8'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:35.790229 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b25eb3bc-cfdf-410d-b15c-b2acb9adf5b8 admin admin] The token cache is being invalidate because project 0e84f6dfc7db4dbc8a0f452e10cf4dec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:26:35.790708 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b25eb3bc-cfdf-410d-b15c-b2acb9adf5b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:35.791450 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b25eb3bc-cfdf-410d-b15c-b2acb9adf5b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:26:35.793553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1215/2413] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:26:35 2022] DELETE /identity/v3/projects/0e84f6dfc7db4dbc8a0f452e10cf4dec => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:35.798360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1941d-d8fc-4888-9192-8d509f73a0c6 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.798890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1941d-d8fc-4888-9192-8d509f73a0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.798890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1941d-d8fc-4888-9192-8d509f73a0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.823739 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9ef1941d-d8fc-4888-9192-8d509f73a0c6 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:35.825008 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9ef1941d-d8fc-4888-9192-8d509f73a0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:35.928139 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1199/2414] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:35.929481 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7cc95a-ef89-402e-8095-611418a358e5 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.929761 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7cc95a-ef89-402e-8095-611418a358e5 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:35.930031 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7cc95a-ef89-402e-8095-611418a358e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:35.965840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-8a7cc95a-ef89-402e-8095-611418a358e5 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:35.967084 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-8a7cc95a-ef89-402e-8095-611418a358e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:35.979819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1216/2415] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:26:35.981904 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1941d-d8fc-4888-9192-8d509f73a0c6 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:35.992744 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1941d-d8fc-4888-9192-8d509f73a0c6 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:35.998753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531c596-ba7b-429a-8d59-d3a9af8f417b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:35.999801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531c596-ba7b-429a-8d59-d3a9af8f417b admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:35.999801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531c596-ba7b-429a-8d59-d3a9af8f417b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:36.000063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531c596-ba7b-429a-8d59-d3a9af8f417b admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:36.000717 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531c596-ba7b-429a-8d59-d3a9af8f417b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:26:36.001010 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531c596-ba7b-429a-8d59-d3a9af8f417b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:36.002840 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531c596-ba7b-429a-8d59-d3a9af8f417b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:36.016866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1200/2416] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:26:36.021095 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7cc95a-ef89-402e-8095-611418a358e5 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:36.031168 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7cc95a-ef89-402e-8095-611418a358e5 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:36.037043 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db88e22-ab25-4a95-a24f-d5ab3c17b904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:36.037769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db88e22-ab25-4a95-a24f-d5ab3c17b904 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:36.038154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db88e22-ab25-4a95-a24f-d5ab3c17b904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:36.038529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db88e22-ab25-4a95-a24f-d5ab3c17b904 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:36.039293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db88e22-ab25-4a95-a24f-d5ab3c17b904 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:36.041321 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db88e22-ab25-4a95-a24f-d5ab3c17b904 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:36.071342 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1217/2417] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:36 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:26:36.075641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531c596-ba7b-429a-8d59-d3a9af8f417b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:36.088892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531c596-ba7b-429a-8d59-d3a9af8f417b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:36.155530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724ecd1-ac7d-430e-922d-ef87ade56d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:36.156169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724ecd1-ac7d-430e-922d-ef87ade56d0c admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:36.156444 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724ecd1-ac7d-430e-922d-ef87ade56d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:36.156720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724ecd1-ac7d-430e-922d-ef87ade56d0c admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:36.157463 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724ecd1-ac7d-430e-922d-ef87ade56d0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:36.159471 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724ecd1-ac7d-430e-922d-ef87ade56d0c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:36.181247 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1201/2418] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:36 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:36.185423 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db88e22-ab25-4a95-a24f-d5ab3c17b904 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:36.197152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db88e22-ab25-4a95-a24f-d5ab3c17b904 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:36.202321 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f67457-ab3c-4b4e-8d1f-22d141a39c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:36.202986 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f67457-ab3c-4b4e-8d1f-22d141a39c32 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:36.203324 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f67457-ab3c-4b4e-8d1f-22d141a39c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:36.203685 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f67457-ab3c-4b4e-8d1f-22d141a39c32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:36.204436 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f67457-ab3c-4b4e-8d1f-22d141a39c32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:36.206114 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f67457-ab3c-4b4e-8d1f-22d141a39c32 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:36.217595 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1218/2419] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:36.221877 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724ecd1-ac7d-430e-922d-ef87ade56d0c admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:36.232284 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724ecd1-ac7d-430e-922d-ef87ade56d0c admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:36.237228 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c8aea-67fe-441c-9247-4f1248386ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:36.237829 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c8aea-67fe-441c-9247-4f1248386ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:36.238114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c8aea-67fe-441c-9247-4f1248386ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:36.238387 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c8aea-67fe-441c-9247-4f1248386ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:36.239117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c8aea-67fe-441c-9247-4f1248386ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:36.240756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c8aea-67fe-441c-9247-4f1248386ea1 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:36.251440 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1202/2420] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:36.256086 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f67457-ab3c-4b4e-8d1f-22d141a39c32 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:36.266397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f67457-ab3c-4b4e-8d1f-22d141a39c32 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:36.272274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f768ea-2865-4204-a28c-cfb74d032d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:36.272966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f768ea-2865-4204-a28c-cfb74d032d18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:36.273307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f768ea-2865-4204-a28c-cfb74d032d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:36.273517 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f768ea-2865-4204-a28c-cfb74d032d18 admin admin] PATH_INFO: `/v3/projects/8351993a1ab94f82bc238a23b2d542d3/users/7f546dc2d8b24eeea8c6eac968c56806/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:36.289705 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f768ea-2865-4204-a28c-cfb74d032d18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8351993a1ab94f82bc238a23b2d542d3, user_id=7f546dc2d8b24eeea8c6eac968c56806, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:36.292531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f768ea-2865-4204-a28c-cfb74d032d18 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:36.309927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1219/2421] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:36 2022] PUT /identity/v3/projects/8351993a1ab94f82bc238a23b2d542d3/users/7f546dc2d8b24eeea8c6eac968c56806/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:36.315051 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855cd3f-cbe6-4250-8e88-abab6bee7bc0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:36.315509 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855cd3f-cbe6-4250-8e88-abab6bee7bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:36.315949 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855cd3f-cbe6-4250-8e88-abab6bee7bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:36.354007 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-1855cd3f-cbe6-4250-8e88-abab6bee7bc0 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:36.355331 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1855cd3f-cbe6-4250-8e88-abab6bee7bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:36.398499 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:36.451038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1203/2422] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:36.462712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f768ea-2865-4204-a28c-cfb74d032d18 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:36.464768 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-62f768ea-2865-4204-a28c-cfb74d032d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:36.535583 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f768ea-2865-4204-a28c-cfb74d032d18 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:36.540433 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea64c1f-f847-42ad-8e84-ae6271f2afd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:36.541091 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea64c1f-f847-42ad-8e84-ae6271f2afd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:36.541379 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea64c1f-f847-42ad-8e84-ae6271f2afd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:36.542052 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea64c1f-f847-42ad-8e84-ae6271f2afd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:36.555750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea64c1f-f847-42ad-8e84-ae6271f2afd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:36.557324 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea64c1f-f847-42ad-8e84-ae6271f2afd9 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:36.561854 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:36.570303 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1220/2423] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:38.367175 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4e78f-3b72-451f-a6ad-0af1ca7b60e5 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:38.367646 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4e78f-3b72-451f-a6ad-0af1ca7b60e5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:38.367741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4e78f-3b72-451f-a6ad-0af1ca7b60e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:38.393280 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-60e4e78f-3b72-451f-a6ad-0af1ca7b60e5 None None] MFA Rules not processed for user `06d279b488d148428b7a3c35a9786ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:38.394645 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-60e4e78f-3b72-451f-a6ad-0af1ca7b60e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:38.461341 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1204/2424] 10.222.0.81 () {58 vars in 967 bytes} [Mon Sep 19 11:26:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:38.480626 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea64c1f-f847-42ad-8e84-ae6271f2afd9 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:38.482103 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-bea64c1f-f847-42ad-8e84-ae6271f2afd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:38.582153 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea64c1f-f847-42ad-8e84-ae6271f2afd9 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:38.588170 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676a6fd-4f65-43d1-8200-527405b2c2ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8a9732807d4921a41a6a2d5019e2f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:38.588778 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676a6fd-4f65-43d1-8200-527405b2c2ee service swift] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:38.589056 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676a6fd-4f65-43d1-8200-527405b2c2ee service swift] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:38.589330 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676a6fd-4f65-43d1-8200-527405b2c2ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:38.598465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676a6fd-4f65-43d1-8200-527405b2c2ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:38.600194 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676a6fd-4f65-43d1-8200-527405b2c2ee service swift] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:38.610263 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1221/2425] 10.222.0.81 () {60 vars in 1384 bytes} [Mon Sep 19 11:26:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:26:41.051860 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e0aed-c3d8-4778-b458-d1ffd50703ee None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:41.052344 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e0aed-c3d8-4778-b458-d1ffd50703ee None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:41.052551 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e0aed-c3d8-4778-b458-d1ffd50703ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:41.090239 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6e7e0aed-c3d8-4778-b458-d1ffd50703ee None None] MFA Rules not processed for user `7f546dc2d8b24eeea8c6eac968c56806`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:41.091506 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6e7e0aed-c3d8-4778-b458-d1ffd50703ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:41.170329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1205/2426] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:26:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:41.171380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676a6fd-4f65-43d1-8200-527405b2c2ee service swift] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:41.184302 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676a6fd-4f65-43d1-8200-527405b2c2ee service swift] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:41.192948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33131dc0-418c-4536-be0a-5acf92293422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:41.193559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33131dc0-418c-4536-be0a-5acf92293422 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:41.193837 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33131dc0-418c-4536-be0a-5acf92293422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:41.194105 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33131dc0-418c-4536-be0a-5acf92293422 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:41.194729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33131dc0-418c-4536-be0a-5acf92293422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:41.196651 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33131dc0-418c-4536-be0a-5acf92293422 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:41.231969 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1222/2427] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:26:41 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:26:41.236103 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e0aed-c3d8-4778-b458-d1ffd50703ee None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:41.247937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e0aed-c3d8-4778-b458-d1ffd50703ee None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:41.315711 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b842f2-3e9b-4774-9729-bd79c13d5be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:41.316460 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b842f2-3e9b-4774-9729-bd79c13d5be3 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:41.317320 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b842f2-3e9b-4774-9729-bd79c13d5be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:41.317660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b842f2-3e9b-4774-9729-bd79c13d5be3 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:41.318409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b842f2-3e9b-4774-9729-bd79c13d5be3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:41.326033 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b842f2-3e9b-4774-9729-bd79c13d5be3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:41.338873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:41.361573 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1206/2428] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:26:41 2022] POST /identity/v3/users => generated 296 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:26:41.362506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33131dc0-418c-4536-be0a-5acf92293422 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:41.379015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33131dc0-418c-4536-be0a-5acf92293422 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:41.388260 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93250d19-427a-428d-a303-6980a4c39d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:41.389082 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93250d19-427a-428d-a303-6980a4c39d72 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:41.389399 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93250d19-427a-428d-a303-6980a4c39d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:41.389772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93250d19-427a-428d-a303-6980a4c39d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:41.390514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93250d19-427a-428d-a303-6980a4c39d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:41.392264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93250d19-427a-428d-a303-6980a4c39d72 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:41.414158 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b842f2-3e9b-4774-9729-bd79c13d5be3 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:41.424912 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1223/2429] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:41.430119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b842f2-3e9b-4774-9729-bd79c13d5be3 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:41.436872 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3b707-e816-4c60-8084-f2cbafd79171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:41.437911 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3b707-e816-4c60-8084-f2cbafd79171 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:41.438287 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3b707-e816-4c60-8084-f2cbafd79171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:41.440183 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3b707-e816-4c60-8084-f2cbafd79171 admin admin] PATH_INFO: `/v3/projects/45c46d122e74432a8228d27ed8b6f876/users/4e87385147af409eb2bde82b870c540e/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:41.459696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3b707-e816-4c60-8084-f2cbafd79171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45c46d122e74432a8228d27ed8b6f876, user_id=4e87385147af409eb2bde82b870c540e, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:41.462549 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3b707-e816-4c60-8084-f2cbafd79171 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:41.484459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1207/2430] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:41 2022] PUT /identity/v3/projects/45c46d122e74432a8228d27ed8b6f876/users/4e87385147af409eb2bde82b870c540e/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:41.488738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93250d19-427a-428d-a303-6980a4c39d72 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:41.501976 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93250d19-427a-428d-a303-6980a4c39d72 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:41.647672 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f891d75-4a3d-4141-a23e-4aac0e7af39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:41.648681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f891d75-4a3d-4141-a23e-4aac0e7af39c admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:41.649155 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f891d75-4a3d-4141-a23e-4aac0e7af39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:41.649490 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f891d75-4a3d-4141-a23e-4aac0e7af39c admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:41.650198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f891d75-4a3d-4141-a23e-4aac0e7af39c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:41.652023 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f891d75-4a3d-4141-a23e-4aac0e7af39c admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:41.674900 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1224/2431] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:26:41.679877 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3b707-e816-4c60-8084-f2cbafd79171 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:41.704591 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3b707-e816-4c60-8084-f2cbafd79171 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:41.738038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00707c38-a216-49a6-bd6e-855e5eefa0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:41.739229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00707c38-a216-49a6-bd6e-855e5eefa0c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:41.739502 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00707c38-a216-49a6-bd6e-855e5eefa0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:41.739801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00707c38-a216-49a6-bd6e-855e5eefa0c7 admin admin] PATH_INFO: `/v3/projects/45c46d122e74432a8228d27ed8b6f876/users/4e87385147af409eb2bde82b870c540e/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:41.742708 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00707c38-a216-49a6-bd6e-855e5eefa0c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45c46d122e74432a8228d27ed8b6f876, user_id=4e87385147af409eb2bde82b870c540e, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:41.745736 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00707c38-a216-49a6-bd6e-855e5eefa0c7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:41.764267 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1208/2432] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:26:41 2022] PUT /identity/v3/projects/45c46d122e74432a8228d27ed8b6f876/users/4e87385147af409eb2bde82b870c540e/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:26:46.516621 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7820dc0-b1ca-4be4-b3a9-28fa16a076da None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:46.517120 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7820dc0-b1ca-4be4-b3a9-28fa16a076da None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:46.517215 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7820dc0-b1ca-4be4-b3a9-28fa16a076da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:46.546861 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-f7820dc0-b1ca-4be4-b3a9-28fa16a076da None None] MFA Rules not processed for user `4e87385147af409eb2bde82b870c540e`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:26:46.548040 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f7820dc0-b1ca-4be4-b3a9-28fa16a076da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:46.590049 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:46.636403 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1225/2433] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:26:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:26:46.650983 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00707c38-a216-49a6-bd6e-855e5eefa0c7 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:26:46.653312 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-00707c38-a216-49a6-bd6e-855e5eefa0c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:26:46.734384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00707c38-a216-49a6-bd6e-855e5eefa0c7 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:26:46.740683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cafb14-5944-4e53-83d2-6df80344f7c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:26:46.741389 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cafb14-5944-4e53-83d2-6df80344f7c2 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:26:46.741675 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cafb14-5944-4e53-83d2-6df80344f7c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:26:46.741951 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cafb14-5944-4e53-83d2-6df80344f7c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:26:46.799469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cafb14-5944-4e53-83d2-6df80344f7c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:26:46.801170 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cafb14-5944-4e53-83d2-6df80344f7c2 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:26:46.809991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1209/2434] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:26:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:26:56.632844 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=11 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:26:56.805882 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=73,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:02.458734 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119767c-2934-414d-af4e-f67e946f0e2a None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:02.459232 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119767c-2934-414d-af4e-f67e946f0e2a None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:02.459326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119767c-2934-414d-af4e-f67e946f0e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:02.478241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-e119767c-2934-414d-af4e-f67e946f0e2a None None] MFA Rules not processed for user `9f160368e1984319ad4fa37c2794a956`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:02.479374 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e119767c-2934-414d-af4e-f67e946f0e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:02.534026 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1226/2435] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:27:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:05.504692 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8191e1b-42f4-4391-8fd6-0498a02b00c0 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:05.505273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8191e1b-42f4-4391-8fd6-0498a02b00c0 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:05.505273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8191e1b-42f4-4391-8fd6-0498a02b00c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:05.528150 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-b8191e1b-42f4-4391-8fd6-0498a02b00c0 None None] MFA Rules not processed for user `7f546dc2d8b24eeea8c6eac968c56806`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:05.529428 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b8191e1b-42f4-4391-8fd6-0498a02b00c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:05.542324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1210/2436] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:27:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:12.526823 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=25 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:15.522139 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:23.286619 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119767c-2934-414d-af4e-f67e946f0e2a None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:27:23.288105 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e119767c-2934-414d-af4e-f67e946f0e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:23.343727 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119767c-2934-414d-af4e-f67e946f0e2a None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:27:23.347429 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb1815-faf5-4e98-b1a5-e1e557c19feb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:27:23.348050 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb1815-faf5-4e98-b1a5-e1e557c19feb service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:23.348325 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb1815-faf5-4e98-b1a5-e1e557c19feb service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:23.348591 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb1815-faf5-4e98-b1a5-e1e557c19feb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:23.357621 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb1815-faf5-4e98-b1a5-e1e557c19feb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:27:23.359030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb1815-faf5-4e98-b1a5-e1e557c19feb service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:27:23.368443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1227/2437] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:27:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:27:30.274328 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008c5e8-36ed-4dd6-9957-99117d268bdf None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:27:30.274845 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008c5e8-36ed-4dd6-9957-99117d268bdf None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:27:30.274943 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008c5e8-36ed-4dd6-9957-99117d268bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:27:30.293688 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e008c5e8-36ed-4dd6-9957-99117d268bdf None None] MFA Rules not processed for user `a3097d9c130746ca840bf85f33bc006b`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:27:30.294902 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e008c5e8-36ed-4dd6-9957-99117d268bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:27:30.348732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1211/2438] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:27:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:27:33.364175 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=26 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:27:40.342283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=25 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:28:00.105672 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f3122-5683-43ce-976a-b2aa6d3fbc75 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:00.106212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f3122-5683-43ce-976a-b2aa6d3fbc75 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:00.106212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f3122-5683-43ce-976a-b2aa6d3fbc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:00.125665 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-861f3122-5683-43ce-976a-b2aa6d3fbc75 None None] MFA Rules not processed for user `9a4a7aca62974db38138a38de399139f`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:00.127450 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-861f3122-5683-43ce-976a-b2aa6d3fbc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:00.181834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1228/2439] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:28:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:28:10.176111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=25 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:28:13.830454 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008c5e8-36ed-4dd6-9957-99117d268bdf None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:13.832099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e008c5e8-36ed-4dd6-9957-99117d268bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:13.898755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008c5e8-36ed-4dd6-9957-99117d268bdf None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:13.903253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656b31c-367e-4ac5-85ba-85d2230f3e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:13.903920 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656b31c-367e-4ac5-85ba-85d2230f3e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:13.904228 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656b31c-367e-4ac5-85ba-85d2230f3e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:13.904599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656b31c-367e-4ac5-85ba-85d2230f3e82 admin admin] PATH_INFO: `/v3/users/9f160368e1984319ad4fa37c2794a956` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:13.905732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656b31c-367e-4ac5-85ba-85d2230f3e82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f160368e1984319ad4fa37c2794a956)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:13.908288 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656b31c-367e-4ac5-85ba-85d2230f3e82 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:13.955491 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0656b31c-367e-4ac5-85ba-85d2230f3e82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f160368e1984319ad4fa37c2794a956'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:13.963164 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0656b31c-367e-4ac5-85ba-85d2230f3e82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f160368e1984319ad4fa37c2794a956'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:13.963805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0656b31c-367e-4ac5-85ba-85d2230f3e82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f160368e1984319ad4fa37c2794a956'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:13.964450 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0656b31c-367e-4ac5-85ba-85d2230f3e82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f160368e1984319ad4fa37c2794a956'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:13.977348 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0656b31c-367e-4ac5-85ba-85d2230f3e82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f160368e1984319ad4fa37c2794a956'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:13.989751 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1212/2440] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:28:13 2022] DELETE /identity/v3/users/9f160368e1984319ad4fa37c2794a956 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:28:14.164210 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f3122-5683-43ce-976a-b2aa6d3fbc75 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:14.176862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f3122-5683-43ce-976a-b2aa6d3fbc75 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:14.261959 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d036e-b8f6-4168-991e-62a0f870477d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:14.262592 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d036e-b8f6-4168-991e-62a0f870477d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:14.262871 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d036e-b8f6-4168-991e-62a0f870477d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:14.263175 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d036e-b8f6-4168-991e-62a0f870477d admin admin] PATH_INFO: `/v3/projects/c7ccdce201f547f5a50c93e27c5e8cb1` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:14.264345 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d036e-b8f6-4168-991e-62a0f870477d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7ccdce201f547f5a50c93e27c5e8cb1)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:14.266839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d036e-b8f6-4168-991e-62a0f870477d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:14.311995 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-822d036e-b8f6-4168-991e-62a0f870477d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7ccdce201f547f5a50c93e27c5e8cb1', 'request_id': 'req-822d036e-b8f6-4168-991e-62a0f870477d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:14.318615 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-822d036e-b8f6-4168-991e-62a0f870477d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7ccdce201f547f5a50c93e27c5e8cb1', 'request_id': 'req-822d036e-b8f6-4168-991e-62a0f870477d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:14.324067 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-822d036e-b8f6-4168-991e-62a0f870477d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7ccdce201f547f5a50c93e27c5e8cb1', 'request_id': 'req-822d036e-b8f6-4168-991e-62a0f870477d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:14.337050 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-822d036e-b8f6-4168-991e-62a0f870477d admin admin] The token cache is being invalidate because project c7ccdce201f547f5a50c93e27c5e8cb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:28:14.337335 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-822d036e-b8f6-4168-991e-62a0f870477d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:14.337970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-822d036e-b8f6-4168-991e-62a0f870477d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:28:14.339444 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1229/2441] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:28:14 2022] DELETE /identity/v3/projects/c7ccdce201f547f5a50c93e27c5e8cb1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:28:14.344484 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7812f-828e-4417-b194-97d257e9bd8e None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:14.344772 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7812f-828e-4417-b194-97d257e9bd8e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:14.345050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7812f-828e-4417-b194-97d257e9bd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:14.364886 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-bfa7812f-828e-4417-b194-97d257e9bd8e None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:14.366087 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bfa7812f-828e-4417-b194-97d257e9bd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:14.424437 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1213/2442] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:28:14.432202 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d57e29-5086-4a0d-88e1-3612a85d3166 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:14.432481 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d57e29-5086-4a0d-88e1-3612a85d3166 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:14.432778 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d57e29-5086-4a0d-88e1-3612a85d3166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:14.452357 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-b4d57e29-5086-4a0d-88e1-3612a85d3166 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:14.454936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b4d57e29-5086-4a0d-88e1-3612a85d3166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:14.462767 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1230/2443] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:28:14.467066 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7812f-828e-4417-b194-97d257e9bd8e None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:14.476890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7812f-828e-4417-b194-97d257e9bd8e None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:14.481424 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b42d306-f66a-4e33-b29d-b4f02efe27bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:14.482014 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42d306-f66a-4e33-b29d-b4f02efe27bb admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:14.482293 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42d306-f66a-4e33-b29d-b4f02efe27bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:14.482566 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42d306-f66a-4e33-b29d-b4f02efe27bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:14.483206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b42d306-f66a-4e33-b29d-b4f02efe27bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:28:14.483508 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b42d306-f66a-4e33-b29d-b4f02efe27bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:14.485790 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b42d306-f66a-4e33-b29d-b4f02efe27bb admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:14.499182 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1214/2444] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:28:14.503632 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d57e29-5086-4a0d-88e1-3612a85d3166 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:14.513075 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d57e29-5086-4a0d-88e1-3612a85d3166 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:14.518308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85991023-74b1-4ee7-8673-0e9e826ccef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:14.518895 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85991023-74b1-4ee7-8673-0e9e826ccef0 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:14.519167 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85991023-74b1-4ee7-8673-0e9e826ccef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:14.519437 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85991023-74b1-4ee7-8673-0e9e826ccef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:14.520126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85991023-74b1-4ee7-8673-0e9e826ccef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:14.521988 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85991023-74b1-4ee7-8673-0e9e826ccef0 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:14.548858 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1231/2445] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:28:14 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:28:14.552984 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b42d306-f66a-4e33-b29d-b4f02efe27bb admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:14.563538 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b42d306-f66a-4e33-b29d-b4f02efe27bb admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:14.567133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:28:14.622969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee87e9e-4b5d-4323-8bfb-77f6b24e538f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:14.623561 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee87e9e-4b5d-4323-8bfb-77f6b24e538f admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:14.623945 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee87e9e-4b5d-4323-8bfb-77f6b24e538f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:14.624224 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee87e9e-4b5d-4323-8bfb-77f6b24e538f admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:14.625530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee87e9e-4b5d-4323-8bfb-77f6b24e538f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:14.627933 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee87e9e-4b5d-4323-8bfb-77f6b24e538f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:14.650396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1215/2446] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:28:14 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:28:14.655051 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85991023-74b1-4ee7-8673-0e9e826ccef0 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:14.666057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85991023-74b1-4ee7-8673-0e9e826ccef0 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:14.670693 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266a290-c3ee-4aaa-a013-86431171e56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:14.671286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266a290-c3ee-4aaa-a013-86431171e56c admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:14.671588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266a290-c3ee-4aaa-a013-86431171e56c admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:14.671891 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266a290-c3ee-4aaa-a013-86431171e56c admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:14.672590 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266a290-c3ee-4aaa-a013-86431171e56c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:14.674163 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266a290-c3ee-4aaa-a013-86431171e56c admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:14.684256 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1232/2447] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:28:14.688223 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee87e9e-4b5d-4323-8bfb-77f6b24e538f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:14.698462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee87e9e-4b5d-4323-8bfb-77f6b24e538f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:14.703223 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aba2a0-689a-48e4-89aa-9bd3614f9ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:14.704380 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aba2a0-689a-48e4-89aa-9bd3614f9ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:14.704909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aba2a0-689a-48e4-89aa-9bd3614f9ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:14.705198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aba2a0-689a-48e4-89aa-9bd3614f9ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:14.705820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aba2a0-689a-48e4-89aa-9bd3614f9ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:14.707416 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aba2a0-689a-48e4-89aa-9bd3614f9ab5 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:14.717946 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1216/2448] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:28:14.722552 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266a290-c3ee-4aaa-a013-86431171e56c admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:14.732482 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266a290-c3ee-4aaa-a013-86431171e56c admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:14.737065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce75eb-1244-4f2b-a0e5-22b5bfc96c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:14.737701 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce75eb-1244-4f2b-a0e5-22b5bfc96c24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:14.737978 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce75eb-1244-4f2b-a0e5-22b5bfc96c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:14.738246 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce75eb-1244-4f2b-a0e5-22b5bfc96c24 admin admin] PATH_INFO: `/v3/projects/0e5ff724699a4dafb37bc5d5e240f0b2/users/c23ac045e3db4798be704a4bf37b3c2c/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:14.753061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce75eb-1244-4f2b-a0e5-22b5bfc96c24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e5ff724699a4dafb37bc5d5e240f0b2, user_id=c23ac045e3db4798be704a4bf37b3c2c, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:14.755679 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce75eb-1244-4f2b-a0e5-22b5bfc96c24 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:14.769161 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1233/2449] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:28:14 2022] PUT /identity/v3/projects/0e5ff724699a4dafb37bc5d5e240f0b2/users/c23ac045e3db4798be704a4bf37b3c2c/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:28:14.774348 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549df41-1782-45f8-b353-35c26f7407f1 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:14.774637 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549df41-1782-45f8-b353-35c26f7407f1 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:14.774912 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549df41-1782-45f8-b353-35c26f7407f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:14.793725 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8549df41-1782-45f8-b353-35c26f7407f1 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:14.794943 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8549df41-1782-45f8-b353-35c26f7407f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:14.853775 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1217/2450] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:28:14.864447 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce75eb-1244-4f2b-a0e5-22b5bfc96c24 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:14.866131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b1ce75eb-1244-4f2b-a0e5-22b5bfc96c24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:14.896577 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:28:14.926299 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce75eb-1244-4f2b-a0e5-22b5bfc96c24 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:14.930024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af13ad0-d248-471f-867d-8eebe4a14db4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:14.930626 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af13ad0-d248-471f-867d-8eebe4a14db4 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:14.930905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af13ad0-d248-471f-867d-8eebe4a14db4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:14.931178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af13ad0-d248-471f-867d-8eebe4a14db4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:14.940188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af13ad0-d248-471f-867d-8eebe4a14db4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:14.941544 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af13ad0-d248-471f-867d-8eebe4a14db4 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:14.950945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1234/2451] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:28:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:28:19.655462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3b44c-1895-4725-a265-fa09d7dd24a6 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:19.655951 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3b44c-1895-4725-a265-fa09d7dd24a6 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:19.656031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3b44c-1895-4725-a265-fa09d7dd24a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:19.715929 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-acc3b44c-1895-4725-a265-fa09d7dd24a6 None None] MFA Rules not processed for user `c23ac045e3db4798be704a4bf37b3c2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:19.717247 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-acc3b44c-1895-4725-a265-fa09d7dd24a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:19.803700 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1218/2452] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:28:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:28:19.821262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af13ad0-d248-471f-867d-8eebe4a14db4 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:19.825421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2af13ad0-d248-471f-867d-8eebe4a14db4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:19.910264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af13ad0-d248-471f-867d-8eebe4a14db4 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:19.916031 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f8255-87ad-4b70-a75f-852d9810c94e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:19.916651 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f8255-87ad-4b70-a75f-852d9810c94e service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:19.916933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f8255-87ad-4b70-a75f-852d9810c94e service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:19.917208 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f8255-87ad-4b70-a75f-852d9810c94e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:19.926536 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f8255-87ad-4b70-a75f-852d9810c94e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:19.928183 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f8255-87ad-4b70-a75f-852d9810c94e service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:19.938710 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1235/2453] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:28:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:28:21.223996 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59149a7c-99d1-4451-b396-5099cd23db6e None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:21.224486 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59149a7c-99d1-4451-b396-5099cd23db6e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:21.224603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59149a7c-99d1-4451-b396-5099cd23db6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:21.245078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-59149a7c-99d1-4451-b396-5099cd23db6e None None] MFA Rules not processed for user `c23ac045e3db4798be704a4bf37b3c2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:21.246285 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-59149a7c-99d1-4451-b396-5099cd23db6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:21.253876 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1219/2454] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:28:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:28:24.287425 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f8255-87ad-4b70-a75f-852d9810c94e service nova] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:24.288886 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5d2f8255-87ad-4b70-a75f-852d9810c94e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:24.301281 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f8255-87ad-4b70-a75f-852d9810c94e service nova] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:24.304869 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e45fb3-cc86-41a7-9b4b-87c892001f09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:24.305471 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e45fb3-cc86-41a7-9b4b-87c892001f09 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:24.305732 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e45fb3-cc86-41a7-9b4b-87c892001f09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:24.306003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e45fb3-cc86-41a7-9b4b-87c892001f09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:24.307673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-80e45fb3-cc86-41a7-9b4b-87c892001f09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:24.321917 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e45fb3-cc86-41a7-9b4b-87c892001f09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:24.323334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e45fb3-cc86-41a7-9b4b-87c892001f09 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:24.334603 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1236/2455] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:28:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:28:26.773116 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59149a7c-99d1-4451-b396-5099cd23db6e None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:26.782650 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59149a7c-99d1-4451-b396-5099cd23db6e None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:26.786164 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cfab70-2081-4f9e-823e-62d6db5a6663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:26.786749 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfab70-2081-4f9e-823e-62d6db5a6663 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:26.787022 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfab70-2081-4f9e-823e-62d6db5a6663 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:26.787289 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfab70-2081-4f9e-823e-62d6db5a6663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:26.788996 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c4cfab70-2081-4f9e-823e-62d6db5a6663 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:26.799735 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:28:26.802324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cfab70-2081-4f9e-823e-62d6db5a6663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:26.804281 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cfab70-2081-4f9e-823e-62d6db5a6663 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:26.816856 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1220/2456] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:28:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:28:34.329661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=54 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:28:36.811286 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:28:42.405167 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431ba92-3494-46f3-a27a-bb136d135351 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:42.405667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431ba92-3494-46f3-a27a-bb136d135351 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:42.405799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431ba92-3494-46f3-a27a-bb136d135351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:42.427271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-2431ba92-3494-46f3-a27a-bb136d135351 None None] MFA Rules not processed for user `06d279b488d148428b7a3c35a9786ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:42.428520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2431ba92-3494-46f3-a27a-bb136d135351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:42.487230 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1237/2457] 10.222.0.81 () {58 vars in 1009 bytes} [Mon Sep 19 11:28:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:28:42.536015 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cfab70-2081-4f9e-823e-62d6db5a6663 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:42.538223 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c4cfab70-2081-4f9e-823e-62d6db5a6663 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:42.610910 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cfab70-2081-4f9e-823e-62d6db5a6663 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:42.616115 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3030f-51c6-4560-b22c-82b5f284ac60 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8a9732807d4921a41a6a2d5019e2f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:42.616732 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3030f-51c6-4560-b22c-82b5f284ac60 service swift] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:42.617012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3030f-51c6-4560-b22c-82b5f284ac60 service swift] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:42.617282 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3030f-51c6-4560-b22c-82b5f284ac60 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:42.626079 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3030f-51c6-4560-b22c-82b5f284ac60 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:42.627699 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3030f-51c6-4560-b22c-82b5f284ac60 service swift] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:42.637186 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1221/2458] 10.222.0.81 () {60 vars in 1384 bytes} [Mon Sep 19 11:28:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:28:44.024849 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431ba92-3494-46f3-a27a-bb136d135351 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:44.048484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431ba92-3494-46f3-a27a-bb136d135351 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:44.055826 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2873bbb-c380-416a-9800-9db22830e407 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:44.056662 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2873bbb-c380-416a-9800-9db22830e407 service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:44.056864 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2873bbb-c380-416a-9800-9db22830e407 service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:44.057144 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2873bbb-c380-416a-9800-9db22830e407 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:44.059063 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f2873bbb-c380-416a-9800-9db22830e407 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:44.148565 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2873bbb-c380-416a-9800-9db22830e407 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:44.151610 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2873bbb-c380-416a-9800-9db22830e407 service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:44.168238 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1238/2459] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:28:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:28:52.085833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3030f-51c6-4560-b22c-82b5f284ac60 service swift] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:52.094959 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3030f-51c6-4560-b22c-82b5f284ac60 service swift] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:52.098887 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83c8d3-814f-4d5b-a92d-6580c84363f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:52.099471 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83c8d3-814f-4d5b-a92d-6580c84363f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:52.099752 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83c8d3-814f-4d5b-a92d-6580c84363f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:52.100086 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83c8d3-814f-4d5b-a92d-6580c84363f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:52.101804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ac83c8d3-814f-4d5b-a92d-6580c84363f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:52.111576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac83c8d3-814f-4d5b-a92d-6580c84363f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:52.113026 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac83c8d3-814f-4d5b-a92d-6580c84363f3 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:52.121002 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1222/2460] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:28:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:28:52.982333 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2873bbb-c380-416a-9800-9db22830e407 service nova] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:52.983786 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f2873bbb-c380-416a-9800-9db22830e407 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:52.998188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2873bbb-c380-416a-9800-9db22830e407 service nova] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:53.004266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f3f9e-4f8b-4cd7-afbc-a21d65346687 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:53.004860 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f3f9e-4f8b-4cd7-afbc-a21d65346687 service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:53.005138 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f3f9e-4f8b-4cd7-afbc-a21d65346687 service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:53.005432 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f3f9e-4f8b-4cd7-afbc-a21d65346687 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:53.007151 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-703f3f9e-4f8b-4cd7-afbc-a21d65346687 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:53.067703 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f3f9e-4f8b-4cd7-afbc-a21d65346687 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:53.069314 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f3f9e-4f8b-4cd7-afbc-a21d65346687 service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:53.080130 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1239/2461] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:28:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:28:56.474466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6912f-30f3-4907-b5e4-aa0d0c053572 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:56.474466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6912f-30f3-4907-b5e4-aa0d0c053572 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:56.475096 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6912f-30f3-4907-b5e4-aa0d0c053572 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:56.476273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1223/2462] 10.222.0.81 () {54 vars in 921 bytes} [Mon Sep 19 11:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:28:56.482220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e4414-6196-40da-9d8c-ab035f684936 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:56.483067 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e4414-6196-40da-9d8c-ab035f684936 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:56.483346 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e4414-6196-40da-9d8c-ab035f684936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:56.506235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-5b4e4414-6196-40da-9d8c-ab035f684936 None None] MFA Rules not processed for user `5abd70a18ccd407bafe3b61c8870986e`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:56.507465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5b4e4414-6196-40da-9d8c-ab035f684936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:56.523996 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1240/2463] 10.222.0.81 () {58 vars in 1018 bytes} [Mon Sep 19 11:28:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:28:56.533210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6912f-30f3-4907-b5e4-aa0d0c053572 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:56.535423 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-37f6912f-30f3-4907-b5e4-aa0d0c053572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:56.610366 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6912f-30f3-4907-b5e4-aa0d0c053572 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:56.614444 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947c2a5-6f06-48f4-900a-cac12ff9b297 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88670b0e1a0f460d8f74917c92db9b9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:56.615006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947c2a5-6f06-48f4-900a-cac12ff9b297 service glance] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:56.615628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947c2a5-6f06-48f4-900a-cac12ff9b297 service glance] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:56.616310 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947c2a5-6f06-48f4-900a-cac12ff9b297 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:56.630724 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947c2a5-6f06-48f4-900a-cac12ff9b297 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:56.632215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947c2a5-6f06-48f4-900a-cac12ff9b297 service glance] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:56.644664 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1224/2464] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:28:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:28:56.752807 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52008d-56cd-4206-9e6e-4debaec36b4e None None] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:56.753199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52008d-56cd-4206-9e6e-4debaec36b4e None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:56.753570 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52008d-56cd-4206-9e6e-4debaec36b4e None None] PATH_INFO: `/` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:56.754958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1241/2465] 10.222.0.81 () {54 vars in 921 bytes} [Mon Sep 19 11:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:28:56.759613 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302187b-4d5f-40f4-8bc2-153ed46a0743 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:56.760018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302187b-4d5f-40f4-8bc2-153ed46a0743 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:56.760302 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302187b-4d5f-40f4-8bc2-153ed46a0743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:56.780223 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f302187b-4d5f-40f4-8bc2-153ed46a0743 None None] MFA Rules not processed for user `65de2e5f67dd4bb6a20cf3934afc8abd`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:56.781964 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f302187b-4d5f-40f4-8bc2-153ed46a0743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:56.788793 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1225/2466] 10.222.0.81 () {58 vars in 1018 bytes} [Mon Sep 19 11:28:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:28:56.801553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52008d-56cd-4206-9e6e-4debaec36b4e None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:56.813784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52008d-56cd-4206-9e6e-4debaec36b4e None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:56.817118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62951a63-0eee-40ab-80ec-7d7986c07879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:56.817708 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62951a63-0eee-40ab-80ec-7d7986c07879 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:56.817979 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62951a63-0eee-40ab-80ec-7d7986c07879 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:56.818227 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62951a63-0eee-40ab-80ec-7d7986c07879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:56.825299 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62951a63-0eee-40ab-80ec-7d7986c07879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:56.826707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62951a63-0eee-40ab-80ec-7d7986c07879 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:56.835189 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1242/2467] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:28:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:28:56.920077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869dc5d-7a2f-4003-93df-8b88dd108033 None None] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:56.920380 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869dc5d-7a2f-4003-93df-8b88dd108033 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:56.920656 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869dc5d-7a2f-4003-93df-8b88dd108033 None None] PATH_INFO: `/` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:56.922848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1226/2468] 10.222.0.81 () {54 vars in 921 bytes} [Mon Sep 19 11:28:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 11:28:56.928372 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111ff65-6e39-4528-9b29-3e6aa33b1fd1 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:56.928656 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111ff65-6e39-4528-9b29-3e6aa33b1fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:56.928928 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111ff65-6e39-4528-9b29-3e6aa33b1fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:56.948402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-3111ff65-6e39-4528-9b29-3e6aa33b1fd1 None None] MFA Rules not processed for user `5abd70a18ccd407bafe3b61c8870986e`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:56.949733 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3111ff65-6e39-4528-9b29-3e6aa33b1fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:56.959578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1243/2469] 10.222.0.81 () {58 vars in 1018 bytes} [Mon Sep 19 11:28:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:28:56.970696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869dc5d-7a2f-4003-93df-8b88dd108033 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:56.982775 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869dc5d-7a2f-4003-93df-8b88dd108033 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:56.987533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c12f7-409d-41d9-8ff1-eab4d3b3813b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:56.988173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c12f7-409d-41d9-8ff1-eab4d3b3813b service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:56.988459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c12f7-409d-41d9-8ff1-eab4d3b3813b service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:56.988731 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c12f7-409d-41d9-8ff1-eab4d3b3813b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:56.999429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c12f7-409d-41d9-8ff1-eab4d3b3813b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:57.000989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c12f7-409d-41d9-8ff1-eab4d3b3813b service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:57.012384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1227/2470] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:28:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:28:57.080740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111ff65-6e39-4528-9b29-3e6aa33b1fd1 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:57.082191 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3111ff65-6e39-4528-9b29-3e6aa33b1fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:57.101986 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:28:57.157740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111ff65-6e39-4528-9b29-3e6aa33b1fd1 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:57.163163 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70370a1f-1d21-43fd-8228-c69722b59461 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e7a18dc4a4d4d35a05f1aafbaaf9c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:57.163794 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70370a1f-1d21-43fd-8228-c69722b59461 service placement] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:57.164119 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70370a1f-1d21-43fd-8228-c69722b59461 service placement] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:57.164359 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70370a1f-1d21-43fd-8228-c69722b59461 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:57.166241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-70370a1f-1d21-43fd-8228-c69722b59461 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:57.178016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70370a1f-1d21-43fd-8228-c69722b59461 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:57.179914 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70370a1f-1d21-43fd-8228-c69722b59461 service placement] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:57.190202 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1244/2471] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:28:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:28:59.026616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f3ad5-3018-449f-8196-e87bd447df2a None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:59.027138 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f3ad5-3018-449f-8196-e87bd447df2a None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:59.027232 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f3ad5-3018-449f-8196-e87bd447df2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:59.046646 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-279f3ad5-3018-449f-8196-e87bd447df2a None None] MFA Rules not processed for user `06d279b488d148428b7a3c35a9786ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:28:59.047908 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-279f3ad5-3018-449f-8196-e87bd447df2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:28:59.056387 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1228/2472] 10.222.0.81 () {58 vars in 1009 bytes} [Mon Sep 19 11:28:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:28:59.074768 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70370a1f-1d21-43fd-8228-c69722b59461 service placement] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:28:59.087171 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70370a1f-1d21-43fd-8228-c69722b59461 service placement] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:28:59.092417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68489678-a98a-42a4-a9a7-3e4b701023fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8a9732807d4921a41a6a2d5019e2f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:28:59.093024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68489678-a98a-42a4-a9a7-3e4b701023fc service swift] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:28:59.093329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68489678-a98a-42a4-a9a7-3e4b701023fc service swift] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:28:59.093701 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68489678-a98a-42a4-a9a7-3e4b701023fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:28:59.102077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68489678-a98a-42a4-a9a7-3e4b701023fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:28:59.103631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68489678-a98a-42a4-a9a7-3e4b701023fc service swift] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:28:59.112320 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1245/2473] 10.222.0.81 () {60 vars in 1384 bytes} [Mon Sep 19 11:28:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:29:09.041426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=76 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:29:09.108552 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=31 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:29:12.813203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f3ad5-3018-449f-8196-e87bd447df2a None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:12.828586 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f3ad5-3018-449f-8196-e87bd447df2a None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:12.834552 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946f4ec-ed92-42a8-a875-13c922c308e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:12.835160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946f4ec-ed92-42a8-a875-13c922c308e6 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:12.835441 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946f4ec-ed92-42a8-a875-13c922c308e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:12.835711 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946f4ec-ed92-42a8-a875-13c922c308e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:12.837520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1946f4ec-ed92-42a8-a875-13c922c308e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:12.850820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1946f4ec-ed92-42a8-a875-13c922c308e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:12.852470 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1946f4ec-ed92-42a8-a875-13c922c308e6 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:12.866610 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1229/2474] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:29:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:29:14.323874 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98930ff4-a41a-4aff-9637-afcf8d03ac79 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:14.324359 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98930ff4-a41a-4aff-9637-afcf8d03ac79 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:14.324461 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98930ff4-a41a-4aff-9637-afcf8d03ac79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:14.343835 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-98930ff4-a41a-4aff-9637-afcf8d03ac79 None None] MFA Rules not processed for user `06d279b488d148428b7a3c35a9786ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:14.345036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-98930ff4-a41a-4aff-9637-afcf8d03ac79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:14.352222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1246/2475] 10.222.0.81 () {58 vars in 967 bytes} [Mon Sep 19 11:29:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:14.365148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946f4ec-ed92-42a8-a875-13c922c308e6 service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:14.376847 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946f4ec-ed92-42a8-a875-13c922c308e6 service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:14.381810 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d8e12-6638-4c65-bd3f-365e6de6edd6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8a9732807d4921a41a6a2d5019e2f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:14.382398 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d8e12-6638-4c65-bd3f-365e6de6edd6 service swift] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:14.382678 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d8e12-6638-4c65-bd3f-365e6de6edd6 service swift] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:14.382953 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d8e12-6638-4c65-bd3f-365e6de6edd6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:14.391062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d8e12-6638-4c65-bd3f-365e6de6edd6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:14.392679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d8e12-6638-4c65-bd3f-365e6de6edd6 service swift] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:14.400577 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1230/2476] 10.222.0.81 () {60 vars in 1384 bytes} [Mon Sep 19 11:29:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:29:24.338969 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:29:24.397384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=12 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:29:28.319191 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848a03c-42cd-41c4-8987-98fd037e2730 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:28.319791 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848a03c-42cd-41c4-8987-98fd037e2730 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:28.320114 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848a03c-42cd-41c4-8987-98fd037e2730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:28.339672 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-c848a03c-42cd-41c4-8987-98fd037e2730 None None] MFA Rules not processed for user `c23ac045e3db4798be704a4bf37b3c2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:28.340914 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c848a03c-42cd-41c4-8987-98fd037e2730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:28.348138 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1247/2477] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:29:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:28.673379 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d8e12-6638-4c65-bd3f-365e6de6edd6 service swift] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:28.682828 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d8e12-6638-4c65-bd3f-365e6de6edd6 service swift] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:28.687137 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1425c42-54cd-45c1-aedc-4552b87535a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:28.687888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1425c42-54cd-45c1-aedc-4552b87535a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:28.688198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1425c42-54cd-45c1-aedc-4552b87535a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:28.688499 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1425c42-54cd-45c1-aedc-4552b87535a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:28.690276 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b1425c42-54cd-45c1-aedc-4552b87535a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:28.700141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1425c42-54cd-45c1-aedc-4552b87535a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:28.701616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1425c42-54cd-45c1-aedc-4552b87535a9 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:28.710469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1231/2478] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:29:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:29:34.465014 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848a03c-42cd-41c4-8987-98fd037e2730 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:34.480678 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848a03c-42cd-41c4-8987-98fd037e2730 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:34.495221 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7271e-be22-4d80-8498-399d0ffac4bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:34.497305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7271e-be22-4d80-8498-399d0ffac4bd service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:34.497587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7271e-be22-4d80-8498-399d0ffac4bd service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:34.498392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7271e-be22-4d80-8498-399d0ffac4bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:34.500398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-8cf7271e-be22-4d80-8498-399d0ffac4bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:34.515348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7271e-be22-4d80-8498-399d0ffac4bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:34.532800 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7271e-be22-4d80-8498-399d0ffac4bd service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:34.574920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1248/2479] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:29:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:29:37.127146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2855235-d08e-4f63-8ca3-a27d4b28ad47 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:37.127146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2855235-d08e-4f63-8ca3-a27d4b28ad47 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:37.127146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2855235-d08e-4f63-8ca3-a27d4b28ad47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:37.145124 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f2855235-d08e-4f63-8ca3-a27d4b28ad47 None None] MFA Rules not processed for user `7f546dc2d8b24eeea8c6eac968c56806`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:37.146341 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f2855235-d08e-4f63-8ca3-a27d4b28ad47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:37.208917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1232/2480] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:29:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:37.609563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7271e-be22-4d80-8498-399d0ffac4bd service nova] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:37.619239 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7271e-be22-4d80-8498-399d0ffac4bd service nova] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:37.624261 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029d758-a2fd-44a6-ad4a-f375ff115c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:37.624871 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029d758-a2fd-44a6-ad4a-f375ff115c3d service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:37.625154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029d758-a2fd-44a6-ad4a-f375ff115c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:37.625430 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029d758-a2fd-44a6-ad4a-f375ff115c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:37.627471 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3029d758-a2fd-44a6-ad4a-f375ff115c3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:37.637350 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029d758-a2fd-44a6-ad4a-f375ff115c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:37.638746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029d758-a2fd-44a6-ad4a-f375ff115c3d service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:37.646582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1249/2481] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:29:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:29:40.869075 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2855235-d08e-4f63-8ca3-a27d4b28ad47 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:40.887595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2855235-d08e-4f63-8ca3-a27d4b28ad47 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:40.894683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d90ff4-fb3d-4fa9-a8f0-df0255df57d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:40.895518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d90ff4-fb3d-4fa9-a8f0-df0255df57d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:40.895969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d90ff4-fb3d-4fa9-a8f0-df0255df57d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:40.897296 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d90ff4-fb3d-4fa9-a8f0-df0255df57d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:40.899641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-92d90ff4-fb3d-4fa9-a8f0-df0255df57d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:40.916616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d90ff4-fb3d-4fa9-a8f0-df0255df57d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:40.918271 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d90ff4-fb3d-4fa9-a8f0-df0255df57d6 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:40.939800 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1233/2482] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:29:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:29:44.118682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029d758-a2fd-44a6-ad4a-f375ff115c3d service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:44.120224 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3029d758-a2fd-44a6-ad4a-f375ff115c3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:44.136768 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029d758-a2fd-44a6-ad4a-f375ff115c3d service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:44.142537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173aa584-8066-4ab5-9080-ea77eeae9d58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e7a18dc4a4d4d35a05f1aafbaaf9c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:44.143482 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173aa584-8066-4ab5-9080-ea77eeae9d58 service placement] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:44.144027 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173aa584-8066-4ab5-9080-ea77eeae9d58 service placement] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:44.144718 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173aa584-8066-4ab5-9080-ea77eeae9d58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:44.146728 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-173aa584-8066-4ab5-9080-ea77eeae9d58 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:44.165997 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173aa584-8066-4ab5-9080-ea77eeae9d58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:44.168954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173aa584-8066-4ab5-9080-ea77eeae9d58 service placement] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:44.185233 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1250/2483] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:29:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:29:44.196708 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d90ff4-fb3d-4fa9-a8f0-df0255df57d6 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:44.197589 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-92d90ff4-fb3d-4fa9-a8f0-df0255df57d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:44.239284 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d90ff4-fb3d-4fa9-a8f0-df0255df57d6 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:44.250019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbb0f5-322e-459d-953e-5bbd9cca904c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:44.250603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbb0f5-322e-459d-953e-5bbd9cca904c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:44.251868 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbb0f5-322e-459d-953e-5bbd9cca904c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:44.252570 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbb0f5-322e-459d-953e-5bbd9cca904c admin admin] PATH_INFO: `/v3/users/9a4a7aca62974db38138a38de399139f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:44.253771 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbb0f5-322e-459d-953e-5bbd9cca904c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a4a7aca62974db38138a38de399139f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:44.255717 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbb0f5-322e-459d-953e-5bbd9cca904c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:44.312762 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5bcbb0f5-322e-459d-953e-5bbd9cca904c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a4a7aca62974db38138a38de399139f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:44.320801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5bcbb0f5-322e-459d-953e-5bbd9cca904c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a4a7aca62974db38138a38de399139f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:44.321797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5bcbb0f5-322e-459d-953e-5bbd9cca904c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a4a7aca62974db38138a38de399139f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:44.322698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5bcbb0f5-322e-459d-953e-5bbd9cca904c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a4a7aca62974db38138a38de399139f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:44.339998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-5bcbb0f5-322e-459d-953e-5bbd9cca904c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a4a7aca62974db38138a38de399139f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:44.353474 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1234/2484] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:29:44 2022] DELETE /identity/v3/users/9a4a7aca62974db38138a38de399139f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:29:44.543316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173aa584-8066-4ab5-9080-ea77eeae9d58 service placement] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:44.554094 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173aa584-8066-4ab5-9080-ea77eeae9d58 service placement] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:44.612395 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cb7ed-96bd-42be-8837-12186c14d79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:44.613042 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cb7ed-96bd-42be-8837-12186c14d79d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:44.613321 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cb7ed-96bd-42be-8837-12186c14d79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:44.613595 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cb7ed-96bd-42be-8837-12186c14d79d admin admin] PATH_INFO: `/v3/projects/e8a952d8324548d7a8e80f72449ea5e6` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:44.614702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431cb7ed-96bd-42be-8837-12186c14d79d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8a952d8324548d7a8e80f72449ea5e6)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:44.616624 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431cb7ed-96bd-42be-8837-12186c14d79d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:44.662966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-431cb7ed-96bd-42be-8837-12186c14d79d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8a952d8324548d7a8e80f72449ea5e6', 'request_id': 'req-431cb7ed-96bd-42be-8837-12186c14d79d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:44.669533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-431cb7ed-96bd-42be-8837-12186c14d79d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8a952d8324548d7a8e80f72449ea5e6', 'request_id': 'req-431cb7ed-96bd-42be-8837-12186c14d79d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:44.675528 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-431cb7ed-96bd-42be-8837-12186c14d79d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8a952d8324548d7a8e80f72449ea5e6', 'request_id': 'req-431cb7ed-96bd-42be-8837-12186c14d79d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:44.688620 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-431cb7ed-96bd-42be-8837-12186c14d79d admin admin] The token cache is being invalidate because project e8a952d8324548d7a8e80f72449ea5e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:29:44.688913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-431cb7ed-96bd-42be-8837-12186c14d79d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:44.689561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-431cb7ed-96bd-42be-8837-12186c14d79d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:44.691178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1251/2485] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:29:44 2022] DELETE /identity/v3/projects/e8a952d8324548d7a8e80f72449ea5e6 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:29:44.696080 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1783778-da55-4833-acb8-c645fd2eceb3 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:44.696365 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1783778-da55-4833-acb8-c645fd2eceb3 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:44.696636 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1783778-da55-4833-acb8-c645fd2eceb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:44.716863 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a1783778-da55-4833-acb8-c645fd2eceb3 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:44.718070 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a1783778-da55-4833-acb8-c645fd2eceb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:44.763791 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:29:44.780239 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1235/2486] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:44.787643 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23befa24-4cfa-4a8b-ad92-087a404e1677 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:44.787956 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23befa24-4cfa-4a8b-ad92-087a404e1677 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:44.788236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23befa24-4cfa-4a8b-ad92-087a404e1677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:44.808413 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-23befa24-4cfa-4a8b-ad92-087a404e1677 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:44.810218 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-23befa24-4cfa-4a8b-ad92-087a404e1677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:44.818825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1252/2487] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:44.822765 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1783778-da55-4833-acb8-c645fd2eceb3 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:44.833517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1783778-da55-4833-acb8-c645fd2eceb3 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:44.838077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc9cc6-8ccb-48f6-a31d-b7431a4cf9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:44.838672 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc9cc6-8ccb-48f6-a31d-b7431a4cf9cf admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:44.838949 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc9cc6-8ccb-48f6-a31d-b7431a4cf9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:44.839219 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc9cc6-8ccb-48f6-a31d-b7431a4cf9cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:44.839896 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc9cc6-8ccb-48f6-a31d-b7431a4cf9cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:29:44.840199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc9cc6-8ccb-48f6-a31d-b7431a4cf9cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:44.841788 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc9cc6-8ccb-48f6-a31d-b7431a4cf9cf admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:44.853366 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1236/2488] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:29:44.857743 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23befa24-4cfa-4a8b-ad92-087a404e1677 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:44.867080 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23befa24-4cfa-4a8b-ad92-087a404e1677 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:44.871921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee55cfc-c238-40d7-8133-610e7c531696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:44.873141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee55cfc-c238-40d7-8133-610e7c531696 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:44.873420 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee55cfc-c238-40d7-8133-610e7c531696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:44.873686 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee55cfc-c238-40d7-8133-610e7c531696 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:44.874303 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee55cfc-c238-40d7-8133-610e7c531696 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:44.876268 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee55cfc-c238-40d7-8133-610e7c531696 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:44.905698 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1253/2489] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:29:44 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:29:44.910004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc9cc6-8ccb-48f6-a31d-b7431a4cf9cf admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:44.920411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc9cc6-8ccb-48f6-a31d-b7431a4cf9cf admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:44.977035 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abefdfb-52e7-44c4-ad62-9d424979c12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:44.977753 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abefdfb-52e7-44c4-ad62-9d424979c12c admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:44.978040 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abefdfb-52e7-44c4-ad62-9d424979c12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:44.978341 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abefdfb-52e7-44c4-ad62-9d424979c12c admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:44.979099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abefdfb-52e7-44c4-ad62-9d424979c12c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:44.981108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abefdfb-52e7-44c4-ad62-9d424979c12c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:45.005114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1237/2490] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:29:44 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:29:45.009710 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee55cfc-c238-40d7-8133-610e7c531696 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:45.021087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee55cfc-c238-40d7-8133-610e7c531696 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:45.025814 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068e9cd-8de1-4f12-a330-84f851e31c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:45.026416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068e9cd-8de1-4f12-a330-84f851e31c02 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:45.026697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068e9cd-8de1-4f12-a330-84f851e31c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:45.026965 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068e9cd-8de1-4f12-a330-84f851e31c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:45.027602 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068e9cd-8de1-4f12-a330-84f851e31c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:45.029218 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068e9cd-8de1-4f12-a330-84f851e31c02 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:45.039527 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1254/2491] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:29:45.043609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abefdfb-52e7-44c4-ad62-9d424979c12c admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:45.054125 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abefdfb-52e7-44c4-ad62-9d424979c12c admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:45.058745 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afd73a-9da7-43d5-991d-ca647c083c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:45.059346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afd73a-9da7-43d5-991d-ca647c083c7b admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:45.060446 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afd73a-9da7-43d5-991d-ca647c083c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:45.060584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afd73a-9da7-43d5-991d-ca647c083c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:45.061154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afd73a-9da7-43d5-991d-ca647c083c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:45.062813 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afd73a-9da7-43d5-991d-ca647c083c7b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:45.074346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1238/2492] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:29:45.078964 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068e9cd-8de1-4f12-a330-84f851e31c02 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:45.085443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:29:45.089291 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068e9cd-8de1-4f12-a330-84f851e31c02 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:45.095537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27769a8c-a5b5-4b51-a49d-92676c3efe91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:45.096386 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27769a8c-a5b5-4b51-a49d-92676c3efe91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:45.096835 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27769a8c-a5b5-4b51-a49d-92676c3efe91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:45.097270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27769a8c-a5b5-4b51-a49d-92676c3efe91 admin admin] PATH_INFO: `/v3/projects/5edd41152b674bb5bb3c24b16fdc6934/users/77b5cd137d7a443b81e928477350c42b/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:45.113628 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27769a8c-a5b5-4b51-a49d-92676c3efe91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5edd41152b674bb5bb3c24b16fdc6934, user_id=77b5cd137d7a443b81e928477350c42b, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:45.116290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27769a8c-a5b5-4b51-a49d-92676c3efe91 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:45.129386 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1255/2493] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:29:45 2022] PUT /identity/v3/projects/5edd41152b674bb5bb3c24b16fdc6934/users/77b5cd137d7a443b81e928477350c42b/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:29:45.140869 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255226c-7ff3-4a77-929d-a4ff78369a01 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:45.141154 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255226c-7ff3-4a77-929d-a4ff78369a01 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:45.141426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255226c-7ff3-4a77-929d-a4ff78369a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:45.169913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-b255226c-7ff3-4a77-929d-a4ff78369a01 None None] MFA Rules not processed for user `77b5cd137d7a443b81e928477350c42b`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:45.171122 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b255226c-7ff3-4a77-929d-a4ff78369a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:45.236957 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1239/2494] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:29:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:45.249506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27769a8c-a5b5-4b51-a49d-92676c3efe91 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:45.251257 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-27769a8c-a5b5-4b51-a49d-92676c3efe91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:45.323185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27769a8c-a5b5-4b51-a49d-92676c3efe91 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:45.329065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f81f7-4e25-4eb5-9b78-bd9d0f4cceb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:45.329604 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f81f7-4e25-4eb5-9b78-bd9d0f4cceb3 service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:45.329883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f81f7-4e25-4eb5-9b78-bd9d0f4cceb3 service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:45.330157 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f81f7-4e25-4eb5-9b78-bd9d0f4cceb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:45.337764 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f81f7-4e25-4eb5-9b78-bd9d0f4cceb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:45.339329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f81f7-4e25-4eb5-9b78-bd9d0f4cceb3 service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:45.348140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1256/2495] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:29:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:29:52.700690 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b255226c-7ff3-4a77-929d-a4ff78369a01 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:52.748027 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:29:52.764813 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b255226c-7ff3-4a77-929d-a4ff78369a01 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:52.769345 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd91783-194f-4f9d-83fa-d49d7bd05e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:52.769969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd91783-194f-4f9d-83fa-d49d7bd05e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:52.770251 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd91783-194f-4f9d-83fa-d49d7bd05e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:52.770560 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd91783-194f-4f9d-83fa-d49d7bd05e04 admin admin] PATH_INFO: `/v3/users/77b5cd137d7a443b81e928477350c42b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:52.771688 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd91783-194f-4f9d-83fa-d49d7bd05e04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77b5cd137d7a443b81e928477350c42b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:52.773646 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd91783-194f-4f9d-83fa-d49d7bd05e04 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:52.819773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fdd91783-194f-4f9d-83fa-d49d7bd05e04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77b5cd137d7a443b81e928477350c42b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:52.827970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fdd91783-194f-4f9d-83fa-d49d7bd05e04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77b5cd137d7a443b81e928477350c42b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:52.828534 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fdd91783-194f-4f9d-83fa-d49d7bd05e04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77b5cd137d7a443b81e928477350c42b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:52.829148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fdd91783-194f-4f9d-83fa-d49d7bd05e04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77b5cd137d7a443b81e928477350c42b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:52.841144 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-fdd91783-194f-4f9d-83fa-d49d7bd05e04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77b5cd137d7a443b81e928477350c42b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:52.853968 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1240/2496] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:29:52 2022] DELETE /identity/v3/users/77b5cd137d7a443b81e928477350c42b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:29:52.858499 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6462d72-d867-4d12-932f-9e9da1b6b1ef None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:52.858968 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6462d72-d867-4d12-932f-9e9da1b6b1ef None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:52.859079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6462d72-d867-4d12-932f-9e9da1b6b1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:52.879286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-b6462d72-d867-4d12-932f-9e9da1b6b1ef None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:52.880799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b6462d72-d867-4d12-932f-9e9da1b6b1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:52.940250 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1257/2497] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:52.951892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd91783-194f-4f9d-83fa-d49d7bd05e04 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:52.953551 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-fdd91783-194f-4f9d-83fa-d49d7bd05e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:53.016963 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd91783-194f-4f9d-83fa-d49d7bd05e04 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:53.020843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59258d47-b8a6-46d4-a732-c003187bac09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:53.021457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59258d47-b8a6-46d4-a732-c003187bac09 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:53.021740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59258d47-b8a6-46d4-a732-c003187bac09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:53.022016 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59258d47-b8a6-46d4-a732-c003187bac09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:53.032424 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59258d47-b8a6-46d4-a732-c003187bac09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:53.033896 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59258d47-b8a6-46d4-a732-c003187bac09 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:53.044028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1241/2498] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:29:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:29:53.438282 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6462d72-d867-4d12-932f-9e9da1b6b1ef None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:53.439865 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b6462d72-d867-4d12-932f-9e9da1b6b1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:53.452277 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6462d72-d867-4d12-932f-9e9da1b6b1ef None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:53.460380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0e808-7c9e-4a37-9ab7-7acf189df074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:53.461204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0e808-7c9e-4a37-9ab7-7acf189df074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:53.461700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0e808-7c9e-4a37-9ab7-7acf189df074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:53.461976 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0e808-7c9e-4a37-9ab7-7acf189df074 admin admin] PATH_INFO: `/v3/projects/5edd41152b674bb5bb3c24b16fdc6934` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:53.463166 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d0e808-7c9e-4a37-9ab7-7acf189df074 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5edd41152b674bb5bb3c24b16fdc6934)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:53.465405 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d0e808-7c9e-4a37-9ab7-7acf189df074 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:53.518765 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a1d0e808-7c9e-4a37-9ab7-7acf189df074 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5edd41152b674bb5bb3c24b16fdc6934', 'request_id': 'req-a1d0e808-7c9e-4a37-9ab7-7acf189df074'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:53.525251 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a1d0e808-7c9e-4a37-9ab7-7acf189df074 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5edd41152b674bb5bb3c24b16fdc6934', 'request_id': 'req-a1d0e808-7c9e-4a37-9ab7-7acf189df074'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:53.530987 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:29:53.532423 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a1d0e808-7c9e-4a37-9ab7-7acf189df074 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5edd41152b674bb5bb3c24b16fdc6934', 'request_id': 'req-a1d0e808-7c9e-4a37-9ab7-7acf189df074'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:53.547632 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a1d0e808-7c9e-4a37-9ab7-7acf189df074 admin admin] The token cache is being invalidate because project 5edd41152b674bb5bb3c24b16fdc6934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:29:53.548069 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a1d0e808-7c9e-4a37-9ab7-7acf189df074 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:53.548710 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a1d0e808-7c9e-4a37-9ab7-7acf189df074 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:53.550582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1258/2499] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:29:53 2022] DELETE /identity/v3/projects/5edd41152b674bb5bb3c24b16fdc6934 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:29:53.555792 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5bf06-ca65-4ddd-bf83-043d9e0de125 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:53.556108 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5bf06-ca65-4ddd-bf83-043d9e0de125 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:53.556382 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5bf06-ca65-4ddd-bf83-043d9e0de125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:53.576488 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d0c5bf06-ca65-4ddd-bf83-043d9e0de125 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:53.578027 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d0c5bf06-ca65-4ddd-bf83-043d9e0de125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:53.642445 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1242/2500] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:53.650206 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c35371-6206-4c6f-b82b-0519c825bc25 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:53.650522 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c35371-6206-4c6f-b82b-0519c825bc25 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:53.650825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c35371-6206-4c6f-b82b-0519c825bc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:53.677494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-46c35371-6206-4c6f-b82b-0519c825bc25 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:53.678721 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-46c35371-6206-4c6f-b82b-0519c825bc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:53.687117 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1259/2501] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:53.691608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5bf06-ca65-4ddd-bf83-043d9e0de125 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:53.701540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5bf06-ca65-4ddd-bf83-043d9e0de125 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:53.709978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d8ded-d798-44f5-b4ce-4e57f7fc68f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:53.710619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d8ded-d798-44f5-b4ce-4e57f7fc68f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:53.711000 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d8ded-d798-44f5-b4ce-4e57f7fc68f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:53.711307 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d8ded-d798-44f5-b4ce-4e57f7fc68f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:53.712050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d8ded-d798-44f5-b4ce-4e57f7fc68f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:29:53.712352 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d8ded-d798-44f5-b4ce-4e57f7fc68f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:53.714040 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d8ded-d798-44f5-b4ce-4e57f7fc68f8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:53.717162 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:29:53.728409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1243/2502] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:29:53.732946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c35371-6206-4c6f-b82b-0519c825bc25 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:53.742632 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c35371-6206-4c6f-b82b-0519c825bc25 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:53.747937 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa57cc7-be2d-4783-b1b1-a1c180120697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:53.748824 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa57cc7-be2d-4783-b1b1-a1c180120697 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:53.749102 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa57cc7-be2d-4783-b1b1-a1c180120697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:53.749374 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa57cc7-be2d-4783-b1b1-a1c180120697 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:53.750026 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa57cc7-be2d-4783-b1b1-a1c180120697 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:53.751978 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa57cc7-be2d-4783-b1b1-a1c180120697 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:53.783748 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1260/2503] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:29:53 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:29:53.785212 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d8ded-d798-44f5-b4ce-4e57f7fc68f8 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:53.796085 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d8ded-d798-44f5-b4ce-4e57f7fc68f8 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:53.854209 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57202d99-ec97-4800-af8d-10d32807625e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:53.854811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57202d99-ec97-4800-af8d-10d32807625e admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:53.855089 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57202d99-ec97-4800-af8d-10d32807625e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:53.855359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57202d99-ec97-4800-af8d-10d32807625e admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:53.856019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57202d99-ec97-4800-af8d-10d32807625e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:53.858647 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57202d99-ec97-4800-af8d-10d32807625e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:53.883512 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1244/2504] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:29:53 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:29:53.884989 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa57cc7-be2d-4783-b1b1-a1c180120697 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:53.898776 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa57cc7-be2d-4783-b1b1-a1c180120697 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:53.903579 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c064fe-55ab-4624-8672-db5e47430416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:53.904829 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c064fe-55ab-4624-8672-db5e47430416 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:53.905178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c064fe-55ab-4624-8672-db5e47430416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:53.905523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c064fe-55ab-4624-8672-db5e47430416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:53.906253 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c064fe-55ab-4624-8672-db5e47430416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:53.907875 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c064fe-55ab-4624-8672-db5e47430416 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:53.918159 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1261/2505] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:29:53.922711 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57202d99-ec97-4800-af8d-10d32807625e admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:53.932851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57202d99-ec97-4800-af8d-10d32807625e admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:53.937547 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ba03b-ce81-48e5-9c79-5966f815f9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:53.938078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ba03b-ce81-48e5-9c79-5966f815f9d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:53.938355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ba03b-ce81-48e5-9c79-5966f815f9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:53.938625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ba03b-ce81-48e5-9c79-5966f815f9d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:53.939262 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ba03b-ce81-48e5-9c79-5966f815f9d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:53.940884 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ba03b-ce81-48e5-9c79-5966f815f9d2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:53.952514 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1245/2506] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:29:53.956628 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c064fe-55ab-4624-8672-db5e47430416 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:53.967575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c064fe-55ab-4624-8672-db5e47430416 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:53.972968 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507cd56-9851-45a7-bfbd-2ccb3bd36210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:53.973601 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507cd56-9851-45a7-bfbd-2ccb3bd36210 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:53.973884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507cd56-9851-45a7-bfbd-2ccb3bd36210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:53.974159 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507cd56-9851-45a7-bfbd-2ccb3bd36210 admin admin] PATH_INFO: `/v3/projects/41c250ab9f55450a83feab4c7a1bccdc/users/55f8f208eb214ffa8acaf1da83ea10f4/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:53.989723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507cd56-9851-45a7-bfbd-2ccb3bd36210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41c250ab9f55450a83feab4c7a1bccdc, user_id=55f8f208eb214ffa8acaf1da83ea10f4, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:53.992411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507cd56-9851-45a7-bfbd-2ccb3bd36210 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:54.008907 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1262/2507] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:29:53 2022] PUT /identity/v3/projects/41c250ab9f55450a83feab4c7a1bccdc/users/55f8f208eb214ffa8acaf1da83ea10f4/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:29:54.020117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32465bfb-eb23-4233-b53b-d598cf8d1a49 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:54.020403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32465bfb-eb23-4233-b53b-d598cf8d1a49 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:54.020704 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32465bfb-eb23-4233-b53b-d598cf8d1a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:54.049083 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-32465bfb-eb23-4233-b53b-d598cf8d1a49 None None] MFA Rules not processed for user `55f8f208eb214ffa8acaf1da83ea10f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:54.050311 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-32465bfb-eb23-4233-b53b-d598cf8d1a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:54.118213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1246/2508] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:29:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:54.129590 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507cd56-9851-45a7-bfbd-2ccb3bd36210 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:54.131286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-8507cd56-9851-45a7-bfbd-2ccb3bd36210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:54.208533 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507cd56-9851-45a7-bfbd-2ccb3bd36210 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:54.215048 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab370d8-46a8-4192-8fd4-459cb6379f27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:54.215673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab370d8-46a8-4192-8fd4-459cb6379f27 service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:54.215962 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab370d8-46a8-4192-8fd4-459cb6379f27 service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:54.216239 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab370d8-46a8-4192-8fd4-459cb6379f27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:54.226676 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab370d8-46a8-4192-8fd4-459cb6379f27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:54.228596 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab370d8-46a8-4192-8fd4-459cb6379f27 service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:54.238056 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1263/2509] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:29:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:29:55.170822 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32465bfb-eb23-4233-b53b-d598cf8d1a49 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:55.172366 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-32465bfb-eb23-4233-b53b-d598cf8d1a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:55.188303 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32465bfb-eb23-4233-b53b-d598cf8d1a49 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:55.193947 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2ec63-6538-42ab-8f0f-ef5eca6643e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:55.194639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2ec63-6538-42ab-8f0f-ef5eca6643e4 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:55.194991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2ec63-6538-42ab-8f0f-ef5eca6643e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:55.195336 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2ec63-6538-42ab-8f0f-ef5eca6643e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:55.197157 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-53a2ec63-6538-42ab-8f0f-ef5eca6643e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:55.209937 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2ec63-6538-42ab-8f0f-ef5eca6643e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:55.211527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2ec63-6538-42ab-8f0f-ef5eca6643e4 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:55.222796 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1247/2510] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:29:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:29:56.934488 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab370d8-46a8-4192-8fd4-459cb6379f27 service nova] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:56.998124 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab370d8-46a8-4192-8fd4-459cb6379f27 service nova] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:57.003565 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24aa7a8-d2ac-4aaa-9799-223fad811cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:57.004252 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24aa7a8-d2ac-4aaa-9799-223fad811cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:57.004534 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24aa7a8-d2ac-4aaa-9799-223fad811cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:57.004806 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24aa7a8-d2ac-4aaa-9799-223fad811cb0 admin admin] PATH_INFO: `/v3/users/55f8f208eb214ffa8acaf1da83ea10f4` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:57.005960 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24aa7a8-d2ac-4aaa-9799-223fad811cb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55f8f208eb214ffa8acaf1da83ea10f4)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:57.007946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24aa7a8-d2ac-4aaa-9799-223fad811cb0 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:57.022549 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:29:57.061254 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e24aa7a8-d2ac-4aaa-9799-223fad811cb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55f8f208eb214ffa8acaf1da83ea10f4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:57.061937 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e24aa7a8-d2ac-4aaa-9799-223fad811cb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55f8f208eb214ffa8acaf1da83ea10f4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:57.062605 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e24aa7a8-d2ac-4aaa-9799-223fad811cb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55f8f208eb214ffa8acaf1da83ea10f4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:57.070496 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e24aa7a8-d2ac-4aaa-9799-223fad811cb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55f8f208eb214ffa8acaf1da83ea10f4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:57.083736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e24aa7a8-d2ac-4aaa-9799-223fad811cb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55f8f208eb214ffa8acaf1da83ea10f4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:57.097337 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1264/2511] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:29:56 2022] DELETE /identity/v3/users/55f8f208eb214ffa8acaf1da83ea10f4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:29:57.102426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b3673-e12c-49d2-bc93-13f4b78c5e17 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:57.102902 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b3673-e12c-49d2-bc93-13f4b78c5e17 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:57.102902 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b3673-e12c-49d2-bc93-13f4b78c5e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:57.122299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-347b3673-e12c-49d2-bc93-13f4b78c5e17 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:57.123529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-347b3673-e12c-49d2-bc93-13f4b78c5e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:57.170388 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:29:57.185918 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1248/2512] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:57.196019 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24aa7a8-d2ac-4aaa-9799-223fad811cb0 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:57.197422 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e24aa7a8-d2ac-4aaa-9799-223fad811cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:57.255935 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24aa7a8-d2ac-4aaa-9799-223fad811cb0 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:57.260309 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c62fa-6636-4232-9cfd-f21bacf9be3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:57.261398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c62fa-6636-4232-9cfd-f21bacf9be3d service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:57.261398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c62fa-6636-4232-9cfd-f21bacf9be3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:57.261813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c62fa-6636-4232-9cfd-f21bacf9be3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:57.273078 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c62fa-6636-4232-9cfd-f21bacf9be3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:57.274727 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c62fa-6636-4232-9cfd-f21bacf9be3d service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:57.288477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1265/2513] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:29:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:29:57.573392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b3673-e12c-49d2-bc93-13f4b78c5e17 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:57.575407 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-347b3673-e12c-49d2-bc93-13f4b78c5e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:57.594167 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b3673-e12c-49d2-bc93-13f4b78c5e17 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:57.600329 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce8166f-8413-485c-a22c-e8be043ced7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:57.601092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8166f-8413-485c-a22c-e8be043ced7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:57.601485 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8166f-8413-485c-a22c-e8be043ced7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:57.601877 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8166f-8413-485c-a22c-e8be043ced7b admin admin] PATH_INFO: `/v3/projects/41c250ab9f55450a83feab4c7a1bccdc` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:57.603117 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce8166f-8413-485c-a22c-e8be043ced7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=41c250ab9f55450a83feab4c7a1bccdc)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:57.605251 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce8166f-8413-485c-a22c-e8be043ced7b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:57.653734 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4ce8166f-8413-485c-a22c-e8be043ced7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41c250ab9f55450a83feab4c7a1bccdc', 'request_id': 'req-4ce8166f-8413-485c-a22c-e8be043ced7b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:57.660947 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4ce8166f-8413-485c-a22c-e8be043ced7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41c250ab9f55450a83feab4c7a1bccdc', 'request_id': 'req-4ce8166f-8413-485c-a22c-e8be043ced7b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:57.666907 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4ce8166f-8413-485c-a22c-e8be043ced7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41c250ab9f55450a83feab4c7a1bccdc', 'request_id': 'req-4ce8166f-8413-485c-a22c-e8be043ced7b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:57.680324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4ce8166f-8413-485c-a22c-e8be043ced7b admin admin] The token cache is being invalidate because project 41c250ab9f55450a83feab4c7a1bccdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:29:57.680617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4ce8166f-8413-485c-a22c-e8be043ced7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:57.681377 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4ce8166f-8413-485c-a22c-e8be043ced7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:57.687297 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1249/2514] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:29:57 2022] DELETE /identity/v3/projects/41c250ab9f55450a83feab4c7a1bccdc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:29:57.689067 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e6452-0849-4bf1-badc-c6a9c4b4dca9 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:57.689364 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e6452-0849-4bf1-badc-c6a9c4b4dca9 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:57.689638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e6452-0849-4bf1-badc-c6a9c4b4dca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:57.711558 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-c79e6452-0849-4bf1-badc-c6a9c4b4dca9 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:57.712992 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c79e6452-0849-4bf1-badc-c6a9c4b4dca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:57.781167 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1266/2515] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:29:57.789816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b508316-9049-4936-9543-6d6b31fb6e95 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:57.790106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b508316-9049-4936-9543-6d6b31fb6e95 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:57.790386 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b508316-9049-4936-9543-6d6b31fb6e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:57.813722 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-5b508316-9049-4936-9543-6d6b31fb6e95 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:57.815316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5b508316-9049-4936-9543-6d6b31fb6e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:57.833274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1250/2516] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:57.837457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e6452-0849-4bf1-badc-c6a9c4b4dca9 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:57.849746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e6452-0849-4bf1-badc-c6a9c4b4dca9 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:57.855528 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ce2f5-dc72-42f6-b3a7-67b51c4928c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:57.856176 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ce2f5-dc72-42f6-b3a7-67b51c4928c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:57.856453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ce2f5-dc72-42f6-b3a7-67b51c4928c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:57.857235 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ce2f5-dc72-42f6-b3a7-67b51c4928c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:57.858101 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ce2f5-dc72-42f6-b3a7-67b51c4928c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:29:57.858403 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ce2f5-dc72-42f6-b3a7-67b51c4928c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:57.860535 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ce2f5-dc72-42f6-b3a7-67b51c4928c5 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:57.881916 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b508316-9049-4936-9543-6d6b31fb6e95 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:57.890860 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1267/2517] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:29:57.896907 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b508316-9049-4936-9543-6d6b31fb6e95 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:57.902705 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f92ea-c96a-4548-90f5-84a1a48da99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:57.903334 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f92ea-c96a-4548-90f5-84a1a48da99a admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:57.903643 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f92ea-c96a-4548-90f5-84a1a48da99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:57.903973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f92ea-c96a-4548-90f5-84a1a48da99a admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:57.904754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f92ea-c96a-4548-90f5-84a1a48da99a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:57.906678 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f92ea-c96a-4548-90f5-84a1a48da99a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:57.951045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1251/2518] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:29:57 2022] POST /identity/v3/projects => generated 363 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:29:57.952726 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ce2f5-dc72-42f6-b3a7-67b51c4928c5 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:57.963982 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ce2f5-dc72-42f6-b3a7-67b51c4928c5 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:58.002856 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:29:58.058077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf07a7e-4212-4a39-9678-85ee05a7596f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:58.058847 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf07a7e-4212-4a39-9678-85ee05a7596f admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:58.059273 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf07a7e-4212-4a39-9678-85ee05a7596f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:58.059755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf07a7e-4212-4a39-9678-85ee05a7596f admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:58.061154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf07a7e-4212-4a39-9678-85ee05a7596f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:58.063206 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf07a7e-4212-4a39-9678-85ee05a7596f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:58.091737 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1268/2519] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:29:57 2022] POST /identity/v3/users => generated 282 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:29:58.095820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f92ea-c96a-4548-90f5-84a1a48da99a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:58.108031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f92ea-c96a-4548-90f5-84a1a48da99a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:58.113841 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4924c9f-e0ee-4ce8-ae7f-490ad30b9cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:58.114030 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4924c9f-e0ee-4ce8-ae7f-490ad30b9cac admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:58.114314 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4924c9f-e0ee-4ce8-ae7f-490ad30b9cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:58.114588 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4924c9f-e0ee-4ce8-ae7f-490ad30b9cac admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:58.115240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4924c9f-e0ee-4ce8-ae7f-490ad30b9cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:58.118929 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4924c9f-e0ee-4ce8-ae7f-490ad30b9cac admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:58.131042 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1252/2520] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:29:58.135165 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf07a7e-4212-4a39-9678-85ee05a7596f admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:58.145609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf07a7e-4212-4a39-9678-85ee05a7596f admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:58.150402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e093d0-d65f-4329-b6c4-9b55dc3990a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:58.151001 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e093d0-d65f-4329-b6c4-9b55dc3990a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:58.151634 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e093d0-d65f-4329-b6c4-9b55dc3990a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:58.151634 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e093d0-d65f-4329-b6c4-9b55dc3990a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:58.152201 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e093d0-d65f-4329-b6c4-9b55dc3990a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:58.154459 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e093d0-d65f-4329-b6c4-9b55dc3990a5 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:58.166809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1269/2521] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:29:58.171721 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4924c9f-e0ee-4ce8-ae7f-490ad30b9cac admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:58.182354 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4924c9f-e0ee-4ce8-ae7f-490ad30b9cac admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:58.187323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1627aae-7c74-44d4-96a0-9b475445f20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:58.188014 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1627aae-7c74-44d4-96a0-9b475445f20d admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:58.188299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1627aae-7c74-44d4-96a0-9b475445f20d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:58.188577 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1627aae-7c74-44d4-96a0-9b475445f20d admin admin] PATH_INFO: `/v3/projects/7e2d251c6ae2409bbd0b464c6b79e1b4/users/2e73ee84350a41b4a0171398daef3d5e/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:58.203619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1627aae-7c74-44d4-96a0-9b475445f20d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e2d251c6ae2409bbd0b464c6b79e1b4, user_id=2e73ee84350a41b4a0171398daef3d5e, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:58.206311 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1627aae-7c74-44d4-96a0-9b475445f20d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:58.220639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1253/2522] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:29:58 2022] PUT /identity/v3/projects/7e2d251c6ae2409bbd0b464c6b79e1b4/users/2e73ee84350a41b4a0171398daef3d5e/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:29:58.231201 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f05ef-1ee7-47c6-bbce-9fb43dc6a892 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:58.231487 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f05ef-1ee7-47c6-bbce-9fb43dc6a892 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:58.232434 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f05ef-1ee7-47c6-bbce-9fb43dc6a892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:58.260146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-e25f05ef-1ee7-47c6-bbce-9fb43dc6a892 None None] MFA Rules not processed for user `2e73ee84350a41b4a0171398daef3d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:58.261357 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e25f05ef-1ee7-47c6-bbce-9fb43dc6a892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:58.327670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1270/2523] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:29:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:58.332372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1627aae-7c74-44d4-96a0-9b475445f20d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:58.399050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1627aae-7c74-44d4-96a0-9b475445f20d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:58.404274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade87b6-84de-4fd8-ba0b-e307eb72c5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:58.404881 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade87b6-84de-4fd8-ba0b-e307eb72c5cf admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:58.405160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade87b6-84de-4fd8-ba0b-e307eb72c5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:58.405434 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade87b6-84de-4fd8-ba0b-e307eb72c5cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:58.406075 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade87b6-84de-4fd8-ba0b-e307eb72c5cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:58.408004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade87b6-84de-4fd8-ba0b-e307eb72c5cf admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:58.425787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:29:58.448453 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1254/2524] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:29:58 2022] POST /identity/v3/projects => generated 363 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:29:58.449612 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f05ef-1ee7-47c6-bbce-9fb43dc6a892 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:58.463757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f05ef-1ee7-47c6-bbce-9fb43dc6a892 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:58.547625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdcd3c-c283-4cca-ac80-b5d477ac71a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:58.548691 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdcd3c-c283-4cca-ac80-b5d477ac71a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:58.549188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdcd3c-c283-4cca-ac80-b5d477ac71a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:58.549682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdcd3c-c283-4cca-ac80-b5d477ac71a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:58.550896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdcd3c-c283-4cca-ac80-b5d477ac71a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:58.554693 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdcd3c-c283-4cca-ac80-b5d477ac71a1 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:58.610999 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1271/2525] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:29:58 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:29:58.617174 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade87b6-84de-4fd8-ba0b-e307eb72c5cf admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:58.630441 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade87b6-84de-4fd8-ba0b-e307eb72c5cf admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:58.637716 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5064b-da94-4be2-a9e7-810561dfd173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:58.639066 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5064b-da94-4be2-a9e7-810561dfd173 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:58.639490 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5064b-da94-4be2-a9e7-810561dfd173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:58.640105 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5064b-da94-4be2-a9e7-810561dfd173 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:58.641125 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5064b-da94-4be2-a9e7-810561dfd173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:58.643737 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5064b-da94-4be2-a9e7-810561dfd173 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:58.664930 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdcd3c-c283-4cca-ac80-b5d477ac71a1 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:58.670499 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1255/2526] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:29:58.676577 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdcd3c-c283-4cca-ac80-b5d477ac71a1 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:58.681923 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b73bdd-89ac-4de8-ae73-651e141548db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:58.682593 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b73bdd-89ac-4de8-ae73-651e141548db admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:58.682871 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b73bdd-89ac-4de8-ae73-651e141548db admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:58.683145 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b73bdd-89ac-4de8-ae73-651e141548db admin admin] PATH_INFO: `/v3/projects/979e3640fb4d4550879e5dba02280f6d/users/474ced2c791e4425a2f05e8f92096fa5/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:58.707081 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b73bdd-89ac-4de8-ae73-651e141548db admin admin] RBAC: Authorizing `identity:create_grant(project_id=979e3640fb4d4550879e5dba02280f6d, user_id=474ced2c791e4425a2f05e8f92096fa5, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:58.710857 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b73bdd-89ac-4de8-ae73-651e141548db admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:58.713748 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:29:58.730491 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1272/2527] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:29:58 2022] PUT /identity/v3/projects/979e3640fb4d4550879e5dba02280f6d/users/474ced2c791e4425a2f05e8f92096fa5/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:29:58.731751 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5064b-da94-4be2-a9e7-810561dfd173 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:58.748276 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5064b-da94-4be2-a9e7-810561dfd173 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:58.808462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015255a-4a49-4cc6-965c-4a7980e5b121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:58.809097 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015255a-4a49-4cc6-965c-4a7980e5b121 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:58.809379 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015255a-4a49-4cc6-965c-4a7980e5b121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:58.809652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015255a-4a49-4cc6-965c-4a7980e5b121 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:58.810334 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015255a-4a49-4cc6-965c-4a7980e5b121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:58.812019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015255a-4a49-4cc6-965c-4a7980e5b121 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:58.822534 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1256/2528] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:29:58.827441 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b73bdd-89ac-4de8-ae73-651e141548db admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:58.838244 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b73bdd-89ac-4de8-ae73-651e141548db admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:58.843792 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a39107-7bf3-4697-937b-1a3a34187eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:58.844460 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a39107-7bf3-4697-937b-1a3a34187eca admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:58.844739 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a39107-7bf3-4697-937b-1a3a34187eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:58.845014 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a39107-7bf3-4697-937b-1a3a34187eca admin admin] PATH_INFO: `/v3/projects/979e3640fb4d4550879e5dba02280f6d/users/474ced2c791e4425a2f05e8f92096fa5/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:58.847069 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a39107-7bf3-4697-937b-1a3a34187eca admin admin] RBAC: Authorizing `identity:create_grant(project_id=979e3640fb4d4550879e5dba02280f6d, user_id=474ced2c791e4425a2f05e8f92096fa5, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:58.849824 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a39107-7bf3-4697-937b-1a3a34187eca admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:58.861074 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1273/2529] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:29:58 2022] PUT /identity/v3/projects/979e3640fb4d4550879e5dba02280f6d/users/474ced2c791e4425a2f05e8f92096fa5/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:29:58.871217 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4af82-8408-44b9-926b-862523024a3d None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:58.871592 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4af82-8408-44b9-926b-862523024a3d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:58.871987 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4af82-8408-44b9-926b-862523024a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:58.902601 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-38e4af82-8408-44b9-926b-862523024a3d None None] MFA Rules not processed for user `474ced2c791e4425a2f05e8f92096fa5`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:58.904014 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-38e4af82-8408-44b9-926b-862523024a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:58.972812 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1257/2530] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:29:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:29:58.985822 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a39107-7bf3-4697-937b-1a3a34187eca admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:58.987750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-18a39107-7bf3-4697-937b-1a3a34187eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:59.049206 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a39107-7bf3-4697-937b-1a3a34187eca admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:59.053637 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0167135-9694-4b0d-8660-f484d904dbb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:59.056531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0167135-9694-4b0d-8660-f484d904dbb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:59.056531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0167135-9694-4b0d-8660-f484d904dbb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:59.056531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0167135-9694-4b0d-8660-f484d904dbb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:59.065419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0167135-9694-4b0d-8660-f484d904dbb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:59.066843 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0167135-9694-4b0d-8660-f484d904dbb5 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:59.076740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1274/2531] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:29:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:29:59.483463 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4af82-8408-44b9-926b-862523024a3d None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:59.545433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4af82-8408-44b9-926b-862523024a3d None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:59.549972 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4185d5f-d652-4db8-bcd6-28b306c37769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:59.550597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4185d5f-d652-4db8-bcd6-28b306c37769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:59.550878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4185d5f-d652-4db8-bcd6-28b306c37769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:59.551181 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4185d5f-d652-4db8-bcd6-28b306c37769 admin admin] PATH_INFO: `/v3/users/2e73ee84350a41b4a0171398daef3d5e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:59.552357 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4185d5f-d652-4db8-bcd6-28b306c37769 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e73ee84350a41b4a0171398daef3d5e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:59.554395 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4185d5f-d652-4db8-bcd6-28b306c37769 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:59.559687 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:29:59.599257 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a4185d5f-d652-4db8-bcd6-28b306c37769 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e73ee84350a41b4a0171398daef3d5e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:59.606103 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a4185d5f-d652-4db8-bcd6-28b306c37769 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e73ee84350a41b4a0171398daef3d5e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:59.607359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a4185d5f-d652-4db8-bcd6-28b306c37769 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e73ee84350a41b4a0171398daef3d5e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:59.608060 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a4185d5f-d652-4db8-bcd6-28b306c37769 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e73ee84350a41b4a0171398daef3d5e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:59.620121 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a4185d5f-d652-4db8-bcd6-28b306c37769 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e73ee84350a41b4a0171398daef3d5e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:29:59.633315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1258/2532] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:29:59 2022] DELETE /identity/v3/users/2e73ee84350a41b4a0171398daef3d5e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:29:59.637765 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6d20e-ac5e-46a1-b642-621ef690bcaa None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:59.638077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6d20e-ac5e-46a1-b642-621ef690bcaa None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:59.638343 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6d20e-ac5e-46a1-b642-621ef690bcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:59.657375 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-9dc6d20e-ac5e-46a1-b642-621ef690bcaa None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:29:59.658562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-9dc6d20e-ac5e-46a1-b642-621ef690bcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:59.717116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1275/2533] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:29:59.726837 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4185d5f-d652-4db8-bcd6-28b306c37769 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:29:59.728388 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a4185d5f-d652-4db8-bcd6-28b306c37769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:29:59.786783 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4185d5f-d652-4db8-bcd6-28b306c37769 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:29:59.791093 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159cf68-896a-451d-8f66-092949294737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:29:59.791819 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159cf68-896a-451d-8f66-092949294737 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:29:59.792173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159cf68-896a-451d-8f66-092949294737 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:29:59.792466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159cf68-896a-451d-8f66-092949294737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:29:59.804728 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159cf68-896a-451d-8f66-092949294737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:29:59.806183 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159cf68-896a-451d-8f66-092949294737 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:29:59.816253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1259/2534] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:29:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:00.188226 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6d20e-ac5e-46a1-b642-621ef690bcaa None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:00.189806 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-9dc6d20e-ac5e-46a1-b642-621ef690bcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:00.202181 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6d20e-ac5e-46a1-b642-621ef690bcaa None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:00.208146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da198c10-6ef6-4309-b6ee-38b22affc5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:00.209055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da198c10-6ef6-4309-b6ee-38b22affc5ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:00.209551 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da198c10-6ef6-4309-b6ee-38b22affc5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:00.210050 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da198c10-6ef6-4309-b6ee-38b22affc5ae admin admin] PATH_INFO: `/v3/projects/7e2d251c6ae2409bbd0b464c6b79e1b4` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:00.211453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da198c10-6ef6-4309-b6ee-38b22affc5ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e2d251c6ae2409bbd0b464c6b79e1b4)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:00.213857 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da198c10-6ef6-4309-b6ee-38b22affc5ae admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:00.266740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-da198c10-6ef6-4309-b6ee-38b22affc5ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e2d251c6ae2409bbd0b464c6b79e1b4', 'request_id': 'req-da198c10-6ef6-4309-b6ee-38b22affc5ae'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:00.273396 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-da198c10-6ef6-4309-b6ee-38b22affc5ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e2d251c6ae2409bbd0b464c6b79e1b4', 'request_id': 'req-da198c10-6ef6-4309-b6ee-38b22affc5ae'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:00.280170 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-da198c10-6ef6-4309-b6ee-38b22affc5ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e2d251c6ae2409bbd0b464c6b79e1b4', 'request_id': 'req-da198c10-6ef6-4309-b6ee-38b22affc5ae'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:00.293833 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-da198c10-6ef6-4309-b6ee-38b22affc5ae admin admin] The token cache is being invalidate because project 7e2d251c6ae2409bbd0b464c6b79e1b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:30:00.294135 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-da198c10-6ef6-4309-b6ee-38b22affc5ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:00.294790 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-da198c10-6ef6-4309-b6ee-38b22affc5ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:00.296377 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1276/2535] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:30:00 2022] DELETE /identity/v3/projects/7e2d251c6ae2409bbd0b464c6b79e1b4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:00.300579 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159cf68-896a-451d-8f66-092949294737 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:00.310124 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159cf68-896a-451d-8f66-092949294737 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:00.368041 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca7390-1894-428e-8ed0-2f9ccacc3157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:00.368785 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca7390-1894-428e-8ed0-2f9ccacc3157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:00.369141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca7390-1894-428e-8ed0-2f9ccacc3157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:00.369487 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca7390-1894-428e-8ed0-2f9ccacc3157 admin admin] PATH_INFO: `/v3/users/474ced2c791e4425a2f05e8f92096fa5` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:00.370640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca7390-1894-428e-8ed0-2f9ccacc3157 admin admin] RBAC: Authorizing `identity:delete_user(user_id=474ced2c791e4425a2f05e8f92096fa5)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:00.373686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca7390-1894-428e-8ed0-2f9ccacc3157 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:00.407274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:00.422785 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8aca7390-1894-428e-8ed0-2f9ccacc3157 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '474ced2c791e4425a2f05e8f92096fa5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:00.430789 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8aca7390-1894-428e-8ed0-2f9ccacc3157 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '474ced2c791e4425a2f05e8f92096fa5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:00.431348 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8aca7390-1894-428e-8ed0-2f9ccacc3157 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '474ced2c791e4425a2f05e8f92096fa5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:00.431997 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8aca7390-1894-428e-8ed0-2f9ccacc3157 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '474ced2c791e4425a2f05e8f92096fa5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:00.445793 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8aca7390-1894-428e-8ed0-2f9ccacc3157 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '474ced2c791e4425a2f05e8f92096fa5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:00.458950 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1260/2536] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:30:00 2022] DELETE /identity/v3/users/474ced2c791e4425a2f05e8f92096fa5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:00.749375 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da198c10-6ef6-4309-b6ee-38b22affc5ae admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:00.750944 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-da198c10-6ef6-4309-b6ee-38b22affc5ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:00.765611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da198c10-6ef6-4309-b6ee-38b22affc5ae admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:00.769506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:00.831269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e015b-e4b1-4348-9856-0412501d5628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:00.831933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e015b-e4b1-4348-9856-0412501d5628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:00.832210 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e015b-e4b1-4348-9856-0412501d5628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:00.832470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e015b-e4b1-4348-9856-0412501d5628 admin admin] PATH_INFO: `/v3/projects/979e3640fb4d4550879e5dba02280f6d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:00.833621 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e015b-e4b1-4348-9856-0412501d5628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=979e3640fb4d4550879e5dba02280f6d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:00.835609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e015b-e4b1-4348-9856-0412501d5628 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:00.884600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5a9e015b-e4b1-4348-9856-0412501d5628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '979e3640fb4d4550879e5dba02280f6d', 'request_id': 'req-5a9e015b-e4b1-4348-9856-0412501d5628'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:00.891803 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5a9e015b-e4b1-4348-9856-0412501d5628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '979e3640fb4d4550879e5dba02280f6d', 'request_id': 'req-5a9e015b-e4b1-4348-9856-0412501d5628'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:00.897270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5a9e015b-e4b1-4348-9856-0412501d5628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '979e3640fb4d4550879e5dba02280f6d', 'request_id': 'req-5a9e015b-e4b1-4348-9856-0412501d5628'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:00.909968 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5a9e015b-e4b1-4348-9856-0412501d5628 admin admin] The token cache is being invalidate because project 979e3640fb4d4550879e5dba02280f6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:30:00.910266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5a9e015b-e4b1-4348-9856-0412501d5628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:00.910902 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5a9e015b-e4b1-4348-9856-0412501d5628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:00.912404 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1277/2537] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:30:00 2022] DELETE /identity/v3/projects/979e3640fb4d4550879e5dba02280f6d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:30:00.917442 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440db2f-8ffb-40a1-abab-516b42d89eee None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:00.917938 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440db2f-8ffb-40a1-abab-516b42d89eee None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:00.918048 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440db2f-8ffb-40a1-abab-516b42d89eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:00.937527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-2440db2f-8ffb-40a1-abab-516b42d89eee None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:00.938745 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2440db2f-8ffb-40a1-abab-516b42d89eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:00.998079 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1261/2538] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:01.005901 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cdde7-703e-4ec9-8885-2977f192a08a None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:01.006185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cdde7-703e-4ec9-8885-2977f192a08a None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:01.006458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cdde7-703e-4ec9-8885-2977f192a08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:01.026410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-364cdde7-703e-4ec9-8885-2977f192a08a None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:01.027644 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-364cdde7-703e-4ec9-8885-2977f192a08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:01.036355 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1278/2539] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:01.040546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440db2f-8ffb-40a1-abab-516b42d89eee None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:01.050427 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440db2f-8ffb-40a1-abab-516b42d89eee None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:01.055198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291824aa-8eec-4de8-8fab-feafa6de0324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:01.055903 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291824aa-8eec-4de8-8fab-feafa6de0324 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:01.056194 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291824aa-8eec-4de8-8fab-feafa6de0324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:01.056476 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291824aa-8eec-4de8-8fab-feafa6de0324 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:01.057144 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291824aa-8eec-4de8-8fab-feafa6de0324 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:30:01.057419 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291824aa-8eec-4de8-8fab-feafa6de0324 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:01.059698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291824aa-8eec-4de8-8fab-feafa6de0324 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:01.071639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1262/2540] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:30:01.076597 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cdde7-703e-4ec9-8885-2977f192a08a None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:01.086301 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cdde7-703e-4ec9-8885-2977f192a08a None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:01.090951 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3c3cc-eadf-4ceb-a915-b70cb4f0c1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:01.092180 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3c3cc-eadf-4ceb-a915-b70cb4f0c1a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:01.092464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3c3cc-eadf-4ceb-a915-b70cb4f0c1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:01.092729 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3c3cc-eadf-4ceb-a915-b70cb4f0c1a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:01.093366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3c3cc-eadf-4ceb-a915-b70cb4f0c1a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:01.095251 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3c3cc-eadf-4ceb-a915-b70cb4f0c1a5 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:01.124638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1279/2541] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:30:01 2022] POST /identity/v3/projects => generated 399 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:30:01.128750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291824aa-8eec-4de8-8fab-feafa6de0324 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:01.141993 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291824aa-8eec-4de8-8fab-feafa6de0324 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:01.197798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ffc36-c9d5-4d34-80bd-8b7cd3d8373f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:01.198316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ffc36-c9d5-4d34-80bd-8b7cd3d8373f admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:01.198632 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ffc36-c9d5-4d34-80bd-8b7cd3d8373f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:01.198911 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ffc36-c9d5-4d34-80bd-8b7cd3d8373f admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:01.199552 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ffc36-c9d5-4d34-80bd-8b7cd3d8373f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:01.202387 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ffc36-c9d5-4d34-80bd-8b7cd3d8373f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:01.222207 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1263/2542] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:30:01 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:30:01.226291 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3c3cc-eadf-4ceb-a915-b70cb4f0c1a5 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:01.236582 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3c3cc-eadf-4ceb-a915-b70cb4f0c1a5 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:01.241281 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513397d-8434-4e9e-b2d2-593787bef7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:01.241877 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513397d-8434-4e9e-b2d2-593787bef7f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:01.242158 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513397d-8434-4e9e-b2d2-593787bef7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:01.242430 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513397d-8434-4e9e-b2d2-593787bef7f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:01.243067 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513397d-8434-4e9e-b2d2-593787bef7f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:01.244698 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513397d-8434-4e9e-b2d2-593787bef7f1 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:01.255159 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1280/2543] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:01.258913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ffc36-c9d5-4d34-80bd-8b7cd3d8373f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:01.269235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ffc36-c9d5-4d34-80bd-8b7cd3d8373f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:01.273918 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9607a6-cb1f-4556-845e-94d630f21704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:01.274497 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9607a6-cb1f-4556-845e-94d630f21704 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:01.274772 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9607a6-cb1f-4556-845e-94d630f21704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:01.275038 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9607a6-cb1f-4556-845e-94d630f21704 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:01.275670 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9607a6-cb1f-4556-845e-94d630f21704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:01.277289 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9607a6-cb1f-4556-845e-94d630f21704 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:01.289953 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1264/2544] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:01.294173 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513397d-8434-4e9e-b2d2-593787bef7f1 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:01.304261 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513397d-8434-4e9e-b2d2-593787bef7f1 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:01.308847 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73d57e-8eb3-44d0-8524-414ff4056fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:01.309477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73d57e-8eb3-44d0-8524-414ff4056fe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:01.310576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73d57e-8eb3-44d0-8524-414ff4056fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:01.310855 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73d57e-8eb3-44d0-8524-414ff4056fe3 admin admin] PATH_INFO: `/v3/projects/f657a78381b94726a59af7ba333f7b4f/users/4938da99322640bd930ff8cfb90835d0/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:01.325373 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73d57e-8eb3-44d0-8524-414ff4056fe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f657a78381b94726a59af7ba333f7b4f, user_id=4938da99322640bd930ff8cfb90835d0, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:01.328830 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73d57e-8eb3-44d0-8524-414ff4056fe3 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:01.341291 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1281/2545] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:01 2022] PUT /identity/v3/projects/f657a78381b94726a59af7ba333f7b4f/users/4938da99322640bd930ff8cfb90835d0/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:01.351582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3aa87-3945-468d-8930-a2880ff436b4 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:01.351924 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3aa87-3945-468d-8930-a2880ff436b4 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:01.352225 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3aa87-3945-468d-8930-a2880ff436b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:01.377405 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:01.380911 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-ccd3aa87-3945-468d-8930-a2880ff436b4 None None] MFA Rules not processed for user `4938da99322640bd930ff8cfb90835d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:01.382779 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ccd3aa87-3945-468d-8930-a2880ff436b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:01.449027 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1265/2546] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:01 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:01.454186 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73d57e-8eb3-44d0-8524-414ff4056fe3 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:01.487716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:01.517104 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73d57e-8eb3-44d0-8524-414ff4056fe3 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:01.521676 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffddc8f-21a0-440b-b3ec-fcc98541ca93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:01.522267 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffddc8f-21a0-440b-b3ec-fcc98541ca93 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:01.522540 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffddc8f-21a0-440b-b3ec-fcc98541ca93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:01.522808 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffddc8f-21a0-440b-b3ec-fcc98541ca93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:01.523425 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffddc8f-21a0-440b-b3ec-fcc98541ca93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:01.525328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffddc8f-21a0-440b-b3ec-fcc98541ca93 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:01.550985 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1282/2547] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:30:01 2022] POST /identity/v3/projects => generated 399 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:30:01.555035 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3aa87-3945-468d-8930-a2880ff436b4 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:01.566168 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3aa87-3945-468d-8930-a2880ff436b4 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:01.622080 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992053f-8193-4787-a057-268108b062f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:01.623273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992053f-8193-4787-a057-268108b062f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:01.623556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992053f-8193-4787-a057-268108b062f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:01.623859 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992053f-8193-4787-a057-268108b062f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:01.624498 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992053f-8193-4787-a057-268108b062f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:01.626410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992053f-8193-4787-a057-268108b062f7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:01.647878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1266/2548] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:30:01 2022] POST /identity/v3/users => generated 306 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:30:01.651919 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffddc8f-21a0-440b-b3ec-fcc98541ca93 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:01.662458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffddc8f-21a0-440b-b3ec-fcc98541ca93 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:01.667126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ebedd-9aad-472a-ad5c-eed3af1b1aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:01.667721 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ebedd-9aad-472a-ad5c-eed3af1b1aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:01.668024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ebedd-9aad-472a-ad5c-eed3af1b1aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:01.668294 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ebedd-9aad-472a-ad5c-eed3af1b1aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:01.668931 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ebedd-9aad-472a-ad5c-eed3af1b1aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:01.671029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ebedd-9aad-472a-ad5c-eed3af1b1aa8 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:01.685740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1283/2549] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:01.687005 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992053f-8193-4787-a057-268108b062f7 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:01.697788 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992053f-8193-4787-a057-268108b062f7 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:01.702465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92d9af-f430-42aa-9e5b-d934a41ef224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:01.703062 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92d9af-f430-42aa-9e5b-d934a41ef224 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:01.703706 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92d9af-f430-42aa-9e5b-d934a41ef224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:01.703706 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92d9af-f430-42aa-9e5b-d934a41ef224 admin admin] PATH_INFO: `/v3/projects/4e80c63312394aeea524052ec55c3e89/users/8f0f884d920741569efe3b2eddf4d9cd/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:01.718839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92d9af-f430-42aa-9e5b-d934a41ef224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e80c63312394aeea524052ec55c3e89, user_id=8f0f884d920741569efe3b2eddf4d9cd, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:01.721535 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92d9af-f430-42aa-9e5b-d934a41ef224 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:01.740438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1267/2550] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:01 2022] PUT /identity/v3/projects/4e80c63312394aeea524052ec55c3e89/users/8f0f884d920741569efe3b2eddf4d9cd/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:01.744363 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ebedd-9aad-472a-ad5c-eed3af1b1aa8 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:01.755748 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ebedd-9aad-472a-ad5c-eed3af1b1aa8 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:01.822314 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d8204-02c0-49a5-ae19-17bf659cee4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:01.823079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d8204-02c0-49a5-ae19-17bf659cee4c admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:01.823364 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d8204-02c0-49a5-ae19-17bf659cee4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:01.823643 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d8204-02c0-49a5-ae19-17bf659cee4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:01.824382 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d8204-02c0-49a5-ae19-17bf659cee4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:01.826176 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d8204-02c0-49a5-ae19-17bf659cee4c admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:01.839849 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1284/2551] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:01.841145 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92d9af-f430-42aa-9e5b-d934a41ef224 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:01.851983 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92d9af-f430-42aa-9e5b-d934a41ef224 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:01.857560 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9be890-fb21-45a9-810b-a88cdf2457d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:01.858216 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9be890-fb21-45a9-810b-a88cdf2457d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:01.858496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9be890-fb21-45a9-810b-a88cdf2457d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:01.858773 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9be890-fb21-45a9-810b-a88cdf2457d6 admin admin] PATH_INFO: `/v3/projects/4e80c63312394aeea524052ec55c3e89/users/8f0f884d920741569efe3b2eddf4d9cd/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:01.860760 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9be890-fb21-45a9-810b-a88cdf2457d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e80c63312394aeea524052ec55c3e89, user_id=8f0f884d920741569efe3b2eddf4d9cd, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:01.863403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9be890-fb21-45a9-810b-a88cdf2457d6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:01.874842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1268/2552] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:01 2022] PUT /identity/v3/projects/4e80c63312394aeea524052ec55c3e89/users/8f0f884d920741569efe3b2eddf4d9cd/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:01.886956 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56ebf5-2bb4-49bd-babc-afe90883ffb8 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:01.887265 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56ebf5-2bb4-49bd-babc-afe90883ffb8 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:01.887552 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56ebf5-2bb4-49bd-babc-afe90883ffb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:01.915698 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-1c56ebf5-2bb4-49bd-babc-afe90883ffb8 None None] MFA Rules not processed for user `8f0f884d920741569efe3b2eddf4d9cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:01.916944 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-1c56ebf5-2bb4-49bd-babc-afe90883ffb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:01.984412 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1285/2553] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:01 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:01.995296 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9be890-fb21-45a9-810b-a88cdf2457d6 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:01.997140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ee9be890-fb21-45a9-810b-a88cdf2457d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:02.055189 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9be890-fb21-45a9-810b-a88cdf2457d6 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:02.058920 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a259845-b890-4f92-83dd-197551fd0c13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:02.059519 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a259845-b890-4f92-83dd-197551fd0c13 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:02.059861 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a259845-b890-4f92-83dd-197551fd0c13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:02.060136 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a259845-b890-4f92-83dd-197551fd0c13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:02.069464 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a259845-b890-4f92-83dd-197551fd0c13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:02.070895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a259845-b890-4f92-83dd-197551fd0c13 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:02.074959 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:02.081215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1269/2554] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:01 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:02.161354 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56ebf5-2bb4-49bd-babc-afe90883ffb8 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:02.171734 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56ebf5-2bb4-49bd-babc-afe90883ffb8 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:02.175344 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a25f17-3bed-404e-9e4c-c4e2441f0b21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:02.175967 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a25f17-3bed-404e-9e4c-c4e2441f0b21 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:02.176246 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a25f17-3bed-404e-9e4c-c4e2441f0b21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:02.176519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a25f17-3bed-404e-9e4c-c4e2441f0b21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:02.202777 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:02.234184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a25f17-3bed-404e-9e4c-c4e2441f0b21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:02.235659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a25f17-3bed-404e-9e4c-c4e2441f0b21 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:02.244391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1286/2555] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:02 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:02.980527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a259845-b890-4f92-83dd-197551fd0c13 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:02.981931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8a259845-b890-4f92-83dd-197551fd0c13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:03.037876 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a259845-b890-4f92-83dd-197551fd0c13 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:03.041466 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566eb90-52bd-498c-9912-a06be601a725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:03.042084 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566eb90-52bd-498c-9912-a06be601a725 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:03.042362 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566eb90-52bd-498c-9912-a06be601a725 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:03.042632 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566eb90-52bd-498c-9912-a06be601a725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:03.044885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b566eb90-52bd-498c-9912-a06be601a725 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:03.106343 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b566eb90-52bd-498c-9912-a06be601a725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:03.108341 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b566eb90-52bd-498c-9912-a06be601a725 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:03.117993 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1270/2556] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:06.383075 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a25f17-3bed-404e-9e4c-c4e2441f0b21 service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:06.384605 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e2a25f17-3bed-404e-9e4c-c4e2441f0b21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:06.395054 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566eb90-52bd-498c-9912-a06be601a725 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:06.405918 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a25f17-3bed-404e-9e4c-c4e2441f0b21 service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:06.410604 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad286e09-7c65-4c4a-86d3-10cc8d6935cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:06.410768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566eb90-52bd-498c-9912-a06be601a725 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:06.411384 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad286e09-7c65-4c4a-86d3-10cc8d6935cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:06.411730 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad286e09-7c65-4c4a-86d3-10cc8d6935cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:06.412115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad286e09-7c65-4c4a-86d3-10cc8d6935cb admin admin] PATH_INFO: `/v3/users/a3097d9c130746ca840bf85f33bc006b` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:06.413339 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad286e09-7c65-4c4a-86d3-10cc8d6935cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3097d9c130746ca840bf85f33bc006b)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:06.415267 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad286e09-7c65-4c4a-86d3-10cc8d6935cb admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:06.415468 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834d930-1803-4248-805e-9f336ff2c5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:06.416190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834d930-1803-4248-805e-9f336ff2c5ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:06.416548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834d930-1803-4248-805e-9f336ff2c5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:06.416904 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834d930-1803-4248-805e-9f336ff2c5ae admin admin] PATH_INFO: `/v3/users/4938da99322640bd930ff8cfb90835d0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:06.418106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834d930-1803-4248-805e-9f336ff2c5ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=4938da99322640bd930ff8cfb90835d0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:06.420198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834d930-1803-4248-805e-9f336ff2c5ae admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:06.466732 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ad286e09-7c65-4c4a-86d3-10cc8d6935cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3097d9c130746ca840bf85f33bc006b'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:06.467296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ad286e09-7c65-4c4a-86d3-10cc8d6935cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3097d9c130746ca840bf85f33bc006b'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:06.467912 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ad286e09-7c65-4c4a-86d3-10cc8d6935cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3097d9c130746ca840bf85f33bc006b'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:06.468818 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3834d930-1803-4248-805e-9f336ff2c5ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4938da99322640bd930ff8cfb90835d0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:06.475571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ad286e09-7c65-4c4a-86d3-10cc8d6935cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3097d9c130746ca840bf85f33bc006b'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:06.477210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3834d930-1803-4248-805e-9f336ff2c5ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4938da99322640bd930ff8cfb90835d0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:06.477918 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3834d930-1803-4248-805e-9f336ff2c5ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4938da99322640bd930ff8cfb90835d0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:06.478707 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3834d930-1803-4248-805e-9f336ff2c5ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4938da99322640bd930ff8cfb90835d0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:06.487425 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ad286e09-7c65-4c4a-86d3-10cc8d6935cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3097d9c130746ca840bf85f33bc006b'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:06.491377 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-3834d930-1803-4248-805e-9f336ff2c5ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4938da99322640bd930ff8cfb90835d0'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:06.500696 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1287/2557] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:30:06 2022] DELETE /identity/v3/users/a3097d9c130746ca840bf85f33bc006b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:06.505916 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1271/2558] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:30:06 2022] DELETE /identity/v3/users/4938da99322640bd930ff8cfb90835d0 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:06.510122 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8167e-120a-434f-9826-3cb4c7867b81 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:06.510405 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8167e-120a-434f-9826-3cb4c7867b81 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:06.510672 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8167e-120a-434f-9826-3cb4c7867b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:06.531529 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-55d8167e-120a-434f-9826-3cb4c7867b81 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:06.532807 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-55d8167e-120a-434f-9826-3cb4c7867b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:06.592953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1288/2559] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:30:06.602581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834d930-1803-4248-805e-9f336ff2c5ae admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:06.604069 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3834d930-1803-4248-805e-9f336ff2c5ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:06.626425 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:06.667661 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834d930-1803-4248-805e-9f336ff2c5ae admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:06.672662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210f96f-89c0-4129-b81b-fdff0b9504c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:06.673521 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210f96f-89c0-4129-b81b-fdff0b9504c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:06.673767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210f96f-89c0-4129-b81b-fdff0b9504c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:06.674045 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210f96f-89c0-4129-b81b-fdff0b9504c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:06.680199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8167e-120a-434f-9826-3cb4c7867b81 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:06.681611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-55d8167e-120a-434f-9826-3cb4c7867b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:06.687226 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210f96f-89c0-4129-b81b-fdff0b9504c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:06.688771 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210f96f-89c0-4129-b81b-fdff0b9504c2 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:06.699298 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1272/2560] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:06.703275 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8167e-120a-434f-9826-3cb4c7867b81 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:06.709635 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47268ef-e462-456d-b583-e02be9feb46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:06.710527 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47268ef-e462-456d-b583-e02be9feb46e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:06.711055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47268ef-e462-456d-b583-e02be9feb46e admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:06.711413 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47268ef-e462-456d-b583-e02be9feb46e admin admin] PATH_INFO: `/v3/projects/078bf59c33cc43579aeb270262079a37` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:06.712676 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47268ef-e462-456d-b583-e02be9feb46e admin admin] RBAC: Authorizing `identity:delete_project(project_id=078bf59c33cc43579aeb270262079a37)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:06.714659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47268ef-e462-456d-b583-e02be9feb46e admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:06.755715 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:06.770593 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b47268ef-e462-456d-b583-e02be9feb46e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '078bf59c33cc43579aeb270262079a37', 'request_id': 'req-b47268ef-e462-456d-b583-e02be9feb46e'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:06.784417 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b47268ef-e462-456d-b583-e02be9feb46e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '078bf59c33cc43579aeb270262079a37', 'request_id': 'req-b47268ef-e462-456d-b583-e02be9feb46e'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:06.791058 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b47268ef-e462-456d-b583-e02be9feb46e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '078bf59c33cc43579aeb270262079a37', 'request_id': 'req-b47268ef-e462-456d-b583-e02be9feb46e'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:06.804188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b47268ef-e462-456d-b583-e02be9feb46e admin admin] The token cache is being invalidate because project 078bf59c33cc43579aeb270262079a37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:30:06.804474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b47268ef-e462-456d-b583-e02be9feb46e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:06.805120 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-b47268ef-e462-456d-b583-e02be9feb46e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:06.806599 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1289/2561] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:30:06 2022] DELETE /identity/v3/projects/078bf59c33cc43579aeb270262079a37 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:30:06.813078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b554b-991b-434c-b843-346693c44d57 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:06.813389 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b554b-991b-434c-b843-346693c44d57 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:06.813679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b554b-991b-434c-b843-346693c44d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:06.834101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6c0b554b-991b-434c-b843-346693c44d57 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:06.835304 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6c0b554b-991b-434c-b843-346693c44d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:06.896058 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1273/2562] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:30:06.904824 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531a08d-5577-4385-8943-5b5fa7a19f14 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:06.905103 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531a08d-5577-4385-8943-5b5fa7a19f14 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:06.905376 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531a08d-5577-4385-8943-5b5fa7a19f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:06.925993 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-8531a08d-5577-4385-8943-5b5fa7a19f14 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:06.927103 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-8531a08d-5577-4385-8943-5b5fa7a19f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:06.935663 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1290/2563] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:06.939874 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b554b-991b-434c-b843-346693c44d57 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:06.949566 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b554b-991b-434c-b843-346693c44d57 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:06.954531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54533e61-a0ae-40f8-b6cb-eab2d3033402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:06.955124 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54533e61-a0ae-40f8-b6cb-eab2d3033402 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:06.955402 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54533e61-a0ae-40f8-b6cb-eab2d3033402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:06.955673 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54533e61-a0ae-40f8-b6cb-eab2d3033402 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:06.956346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54533e61-a0ae-40f8-b6cb-eab2d3033402 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:30:06.956655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54533e61-a0ae-40f8-b6cb-eab2d3033402 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:06.958264 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54533e61-a0ae-40f8-b6cb-eab2d3033402 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:06.971815 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1274/2564] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:30:06.974230 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531a08d-5577-4385-8943-5b5fa7a19f14 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:06.986317 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531a08d-5577-4385-8943-5b5fa7a19f14 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:06.989925 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54533e61-a0ae-40f8-b6cb-eab2d3033402 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:06.991830 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2693c-94e7-422e-a307-62561393754c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:06.991990 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-54533e61-a0ae-40f8-b6cb-eab2d3033402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:06.992443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2693c-94e7-422e-a307-62561393754c admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:06.992726 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2693c-94e7-422e-a307-62561393754c admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:06.993003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2693c-94e7-422e-a307-62561393754c admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:06.994015 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2693c-94e7-422e-a307-62561393754c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:06.995677 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2693c-94e7-422e-a307-62561393754c admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:07.009043 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54533e61-a0ae-40f8-b6cb-eab2d3033402 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:07.016396 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bcfa7-e51f-49dd-92cb-eea3c232d954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:07.016946 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bcfa7-e51f-49dd-92cb-eea3c232d954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:07.017294 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bcfa7-e51f-49dd-92cb-eea3c232d954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:07.017520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6bcfa7-e51f-49dd-92cb-eea3c232d954 admin admin] PATH_INFO: `/v3/projects/f657a78381b94726a59af7ba333f7b4f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:07.019135 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6bcfa7-e51f-49dd-92cb-eea3c232d954 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f657a78381b94726a59af7ba333f7b4f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:07.021357 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6bcfa7-e51f-49dd-92cb-eea3c232d954 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:07.031652 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1291/2565] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:30:06 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:30:07.033076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2693c-94e7-422e-a307-62561393754c admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:07.102743 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2693c-94e7-422e-a307-62561393754c admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:07.108176 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a0f98-dc38-43a3-b035-8d771d924135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:07.108799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0f98-dc38-43a3-b035-8d771d924135 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:07.109084 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0f98-dc38-43a3-b035-8d771d924135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:07.109354 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0f98-dc38-43a3-b035-8d771d924135 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:07.109996 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a0f98-dc38-43a3-b035-8d771d924135 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:07.111954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a0f98-dc38-43a3-b035-8d771d924135 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:07.139691 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1292/2566] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:30:07 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:30:07.140822 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a0f98-dc38-43a3-b035-8d771d924135 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:07.151354 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a0f98-dc38-43a3-b035-8d771d924135 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:07.156887 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f1a8d-f05c-4074-94c2-a5b539944e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:07.157488 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f1a8d-f05c-4074-94c2-a5b539944e5d admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:07.157768 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f1a8d-f05c-4074-94c2-a5b539944e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:07.158045 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f1a8d-f05c-4074-94c2-a5b539944e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:07.158692 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f1a8d-f05c-4074-94c2-a5b539944e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:07.160379 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f1a8d-f05c-4074-94c2-a5b539944e5d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:07.171625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1293/2567] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:07.175565 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f1a8d-f05c-4074-94c2-a5b539944e5d admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:07.185555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f1a8d-f05c-4074-94c2-a5b539944e5d admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:07.194344 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-dc6bcfa7-e51f-49dd-92cb-eea3c232d954 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f657a78381b94726a59af7ba333f7b4f', 'request_id': 'req-dc6bcfa7-e51f-49dd-92cb-eea3c232d954'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:07.201256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-dc6bcfa7-e51f-49dd-92cb-eea3c232d954 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f657a78381b94726a59af7ba333f7b4f', 'request_id': 'req-dc6bcfa7-e51f-49dd-92cb-eea3c232d954'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:07.207375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-dc6bcfa7-e51f-49dd-92cb-eea3c232d954 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f657a78381b94726a59af7ba333f7b4f', 'request_id': 'req-dc6bcfa7-e51f-49dd-92cb-eea3c232d954'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:07.220359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-dc6bcfa7-e51f-49dd-92cb-eea3c232d954 admin admin] The token cache is being invalidate because project f657a78381b94726a59af7ba333f7b4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:30:07.220709 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-dc6bcfa7-e51f-49dd-92cb-eea3c232d954 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:07.221361 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-dc6bcfa7-e51f-49dd-92cb-eea3c232d954 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:07.222908 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1275/2568] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:30:06 2022] DELETE /identity/v3/projects/f657a78381b94726a59af7ba333f7b4f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:30:07.226839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bcfa7-e51f-49dd-92cb-eea3c232d954 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:07.228208 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-dc6bcfa7-e51f-49dd-92cb-eea3c232d954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:07.245434 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625269eb-9d0d-4db9-a935-9e21d08edca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:07.246036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625269eb-9d0d-4db9-a935-9e21d08edca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:07.246316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625269eb-9d0d-4db9-a935-9e21d08edca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:07.246588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625269eb-9d0d-4db9-a935-9e21d08edca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:07.247255 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625269eb-9d0d-4db9-a935-9e21d08edca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:07.248863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625269eb-9d0d-4db9-a935-9e21d08edca7 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:07.259796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1294/2569] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:30:07.262623 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625269eb-9d0d-4db9-a935-9e21d08edca7 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:07.264316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-625269eb-9d0d-4db9-a935-9e21d08edca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:07.266605 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:07.276555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625269eb-9d0d-4db9-a935-9e21d08edca7 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:07.281983 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34170e3b-1fda-4049-9472-bfd2249da5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:07.282630 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34170e3b-1fda-4049-9472-bfd2249da5aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:07.282916 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34170e3b-1fda-4049-9472-bfd2249da5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:07.283193 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34170e3b-1fda-4049-9472-bfd2249da5aa admin admin] PATH_INFO: `/v3/projects/89daac7d8a974f2783682b14b92920ac/users/3d5b88cce500405b8a6041ff44fee4ac/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:07.286430 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:07.300435 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34170e3b-1fda-4049-9472-bfd2249da5aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=89daac7d8a974f2783682b14b92920ac, user_id=3d5b88cce500405b8a6041ff44fee4ac, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:07.300746 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6bcfa7-e51f-49dd-92cb-eea3c232d954 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:07.303437 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34170e3b-1fda-4049-9472-bfd2249da5aa admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:07.305321 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f244a4-1031-4807-881a-ed5df71e6e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:07.305971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f244a4-1031-4807-881a-ed5df71e6e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:07.306249 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f244a4-1031-4807-881a-ed5df71e6e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:07.306523 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f244a4-1031-4807-881a-ed5df71e6e53 admin admin] PATH_INFO: `/v3/users/8f0f884d920741569efe3b2eddf4d9cd` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:07.307640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f244a4-1031-4807-881a-ed5df71e6e53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f0f884d920741569efe3b2eddf4d9cd)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:07.309917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f244a4-1031-4807-881a-ed5df71e6e53 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:07.317832 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1295/2570] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:07 2022] PUT /identity/v3/projects/89daac7d8a974f2783682b14b92920ac/users/3d5b88cce500405b8a6041ff44fee4ac/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:30:07.328628 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba9fc4-8c9e-485b-abdf-2b9ecf0c023d None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:07.328910 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba9fc4-8c9e-485b-abdf-2b9ecf0c023d None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:07.329180 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba9fc4-8c9e-485b-abdf-2b9ecf0c023d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:07.359309 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-53f244a4-1031-4807-881a-ed5df71e6e53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f0f884d920741569efe3b2eddf4d9cd'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:07.359975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-aaba9fc4-8c9e-485b-abdf-2b9ecf0c023d None None] MFA Rules not processed for user `3d5b88cce500405b8a6041ff44fee4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:07.361234 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-aaba9fc4-8c9e-485b-abdf-2b9ecf0c023d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:07.366613 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-53f244a4-1031-4807-881a-ed5df71e6e53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f0f884d920741569efe3b2eddf4d9cd'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:07.367614 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-53f244a4-1031-4807-881a-ed5df71e6e53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f0f884d920741569efe3b2eddf4d9cd'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:07.368327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-53f244a4-1031-4807-881a-ed5df71e6e53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f0f884d920741569efe3b2eddf4d9cd'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:07.380946 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-53f244a4-1031-4807-881a-ed5df71e6e53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f0f884d920741569efe3b2eddf4d9cd'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:07.394342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1276/2571] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:30:07 2022] DELETE /identity/v3/users/8f0f884d920741569efe3b2eddf4d9cd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:30:07.430539 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1296/2572] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:30:07.435550 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f244a4-1031-4807-881a-ed5df71e6e53 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:07.437273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-53f244a4-1031-4807-881a-ed5df71e6e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:07.516712 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f244a4-1031-4807-881a-ed5df71e6e53 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:07.521750 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58676a-7e9e-4685-a3f5-14f3fa1bff8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:07.522623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58676a-7e9e-4685-a3f5-14f3fa1bff8c admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:07.522904 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58676a-7e9e-4685-a3f5-14f3fa1bff8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:07.523173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58676a-7e9e-4685-a3f5-14f3fa1bff8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:07.523969 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58676a-7e9e-4685-a3f5-14f3fa1bff8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:07.525883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58676a-7e9e-4685-a3f5-14f3fa1bff8c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:07.557658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1277/2573] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:30:07 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:30:07.562276 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba9fc4-8c9e-485b-abdf-2b9ecf0c023d None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:07.627073 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba9fc4-8c9e-485b-abdf-2b9ecf0c023d None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:07.691550 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456d2cc-e379-4b46-8bd1-4a44b0dc12fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:07.692393 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456d2cc-e379-4b46-8bd1-4a44b0dc12fc admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:07.692767 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456d2cc-e379-4b46-8bd1-4a44b0dc12fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:07.693140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456d2cc-e379-4b46-8bd1-4a44b0dc12fc admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:07.694069 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456d2cc-e379-4b46-8bd1-4a44b0dc12fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:07.696589 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456d2cc-e379-4b46-8bd1-4a44b0dc12fc admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:07.697349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58676a-7e9e-4685-a3f5-14f3fa1bff8c admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:07.699203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-dd58676a-7e9e-4685-a3f5-14f3fa1bff8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:07.713437 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58676a-7e9e-4685-a3f5-14f3fa1bff8c admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:07.719333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449ebeb-b80e-49f5-ad83-fc7988983f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:07.723079 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449ebeb-b80e-49f5-ad83-fc7988983f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:07.723079 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449ebeb-b80e-49f5-ad83-fc7988983f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:07.723315 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1297/2574] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:30:07 2022] POST /identity/v3/users => generated 295 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:30:07.723420 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449ebeb-b80e-49f5-ad83-fc7988983f04 admin admin] PATH_INFO: `/v3/projects/4e80c63312394aeea524052ec55c3e89` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:07.723420 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449ebeb-b80e-49f5-ad83-fc7988983f04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e80c63312394aeea524052ec55c3e89)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:07.724044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449ebeb-b80e-49f5-ad83-fc7988983f04 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:07.724757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456d2cc-e379-4b46-8bd1-4a44b0dc12fc admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:07.737739 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456d2cc-e379-4b46-8bd1-4a44b0dc12fc admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:07.742557 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be14e0ad-2953-4258-ab79-b2169aa310d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:07.743149 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14e0ad-2953-4258-ab79-b2169aa310d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:07.743426 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14e0ad-2953-4258-ab79-b2169aa310d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:07.743700 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14e0ad-2953-4258-ab79-b2169aa310d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:07.744368 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be14e0ad-2953-4258-ab79-b2169aa310d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:07.745940 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be14e0ad-2953-4258-ab79-b2169aa310d1 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:07.752271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:07.757843 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1298/2575] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:30:07.761696 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be14e0ad-2953-4258-ab79-b2169aa310d1 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:07.764865 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:07.771681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be14e0ad-2953-4258-ab79-b2169aa310d1 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:07.779335 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0449ebeb-b80e-49f5-ad83-fc7988983f04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e80c63312394aeea524052ec55c3e89', 'request_id': 'req-0449ebeb-b80e-49f5-ad83-fc7988983f04'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:07.786029 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0449ebeb-b80e-49f5-ad83-fc7988983f04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e80c63312394aeea524052ec55c3e89', 'request_id': 'req-0449ebeb-b80e-49f5-ad83-fc7988983f04'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:07.792333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0449ebeb-b80e-49f5-ad83-fc7988983f04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e80c63312394aeea524052ec55c3e89', 'request_id': 'req-0449ebeb-b80e-49f5-ad83-fc7988983f04'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:07.806091 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0449ebeb-b80e-49f5-ad83-fc7988983f04 admin admin] The token cache is being invalidate because project 4e80c63312394aeea524052ec55c3e89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:30:07.806618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0449ebeb-b80e-49f5-ad83-fc7988983f04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:07.807430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0449ebeb-b80e-49f5-ad83-fc7988983f04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:07.808889 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1278/2576] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:30:07 2022] DELETE /identity/v3/projects/4e80c63312394aeea524052ec55c3e89 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:07.814173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a297b-114e-45ea-b1cd-e536b116c147 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:07.814461 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a297b-114e-45ea-b1cd-e536b116c147 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:07.814742 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a297b-114e-45ea-b1cd-e536b116c147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:07.831443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227d3e3-1b33-4bd4-aaae-d7eed4a31e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:07.832155 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227d3e3-1b33-4bd4-aaae-d7eed4a31e02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:07.832434 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227d3e3-1b33-4bd4-aaae-d7eed4a31e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:07.832705 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227d3e3-1b33-4bd4-aaae-d7eed4a31e02 admin admin] PATH_INFO: `/v3/projects/ea46ef8ffbdd41ce9192ce9cb80ddef6/users/d9a46846eb064c73829e965dab71b88a/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:07.834384 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-fe3a297b-114e-45ea-b1cd-e536b116c147 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:07.836359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-fe3a297b-114e-45ea-b1cd-e536b116c147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:07.843648 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1279/2577] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:07.847330 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227d3e3-1b33-4bd4-aaae-d7eed4a31e02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea46ef8ffbdd41ce9192ce9cb80ddef6, user_id=d9a46846eb064c73829e965dab71b88a, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:07.850118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227d3e3-1b33-4bd4-aaae-d7eed4a31e02 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:07.851179 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e713ef-3431-48aa-a796-61a3f30b99d2 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:07.851465 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e713ef-3431-48aa-a796-61a3f30b99d2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:07.851740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e713ef-3431-48aa-a796-61a3f30b99d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:07.863625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1299/2578] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:07 2022] PUT /identity/v3/projects/ea46ef8ffbdd41ce9192ce9cb80ddef6/users/d9a46846eb064c73829e965dab71b88a/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:30:07.867208 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227d3e3-1b33-4bd4-aaae-d7eed4a31e02 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:07.868939 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f227d3e3-1b33-4bd4-aaae-d7eed4a31e02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:07.872515 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-29e713ef-3431-48aa-a796-61a3f30b99d2 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:07.873856 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-29e713ef-3431-48aa-a796-61a3f30b99d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:07.933397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1280/2579] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:30:07.934085 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227d3e3-1b33-4bd4-aaae-d7eed4a31e02 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:07.937726 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e713ef-3431-48aa-a796-61a3f30b99d2 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:07.939087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d448e-2223-46f0-a990-2056b3abbaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:07.939745 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d448e-2223-46f0-a990-2056b3abbaa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:07.940005 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d448e-2223-46f0-a990-2056b3abbaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:07.940278 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d448e-2223-46f0-a990-2056b3abbaa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:07.940948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d448e-2223-46f0-a990-2056b3abbaa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:07.942573 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d448e-2223-46f0-a990-2056b3abbaa3 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:07.946399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e713ef-3431-48aa-a796-61a3f30b99d2 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:07.951386 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f8796-4215-4e64-b5f5-277e16e5cfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:07.952270 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f8796-4215-4e64-b5f5-277e16e5cfba admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:07.952546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f8796-4215-4e64-b5f5-277e16e5cfba admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:07.953114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f8796-4215-4e64-b5f5-277e16e5cfba admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:07.953479 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f8796-4215-4e64-b5f5-277e16e5cfba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:30:07.953780 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f8796-4215-4e64-b5f5-277e16e5cfba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:07.955403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f8796-4215-4e64-b5f5-277e16e5cfba admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:07.960822 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1300/2580] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:30:07.961905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d448e-2223-46f0-a990-2056b3abbaa3 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:07.968478 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1281/2581] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:30:07.972691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f8796-4215-4e64-b5f5-277e16e5cfba admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:07.972820 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d448e-2223-46f0-a990-2056b3abbaa3 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:07.977669 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2337d-2738-4aed-9ca2-841fdb566ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:07.978346 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2337d-2738-4aed-9ca2-841fdb566ed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:07.979224 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2337d-2738-4aed-9ca2-841fdb566ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:07.979525 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2337d-2738-4aed-9ca2-841fdb566ed2 admin admin] PATH_INFO: `/v3/projects/ea46ef8ffbdd41ce9192ce9cb80ddef6/users/d9a46846eb064c73829e965dab71b88a/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:07.982067 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2337d-2738-4aed-9ca2-841fdb566ed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea46ef8ffbdd41ce9192ce9cb80ddef6, user_id=d9a46846eb064c73829e965dab71b88a, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:07.982944 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f8796-4215-4e64-b5f5-277e16e5cfba admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:07.984750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2337d-2738-4aed-9ca2-841fdb566ed2 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:07.987956 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc586d5-09b2-4d2e-8a11-ecfd84cf08c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:07.988533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc586d5-09b2-4d2e-8a11-ecfd84cf08c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:07.988807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc586d5-09b2-4d2e-8a11-ecfd84cf08c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:07.989077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc586d5-09b2-4d2e-8a11-ecfd84cf08c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:07.989694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc586d5-09b2-4d2e-8a11-ecfd84cf08c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:07.991551 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc586d5-09b2-4d2e-8a11-ecfd84cf08c5 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:07.995355 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1301/2582] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:07 2022] PUT /identity/v3/projects/ea46ef8ffbdd41ce9192ce9cb80ddef6/users/d9a46846eb064c73829e965dab71b88a/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:08.005681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132480e-36a1-4354-9f05-981f284eed95 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:08.006033 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132480e-36a1-4354-9f05-981f284eed95 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:08.006379 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132480e-36a1-4354-9f05-981f284eed95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:08.020198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1282/2583] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:30:07 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:30:08.024489 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc586d5-09b2-4d2e-8a11-ecfd84cf08c5 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:08.035258 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-e132480e-36a1-4354-9f05-981f284eed95 None None] MFA Rules not processed for user `d9a46846eb064c73829e965dab71b88a`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:08.036513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e132480e-36a1-4354-9f05-981f284eed95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:08.087963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:08.088623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc586d5-09b2-4d2e-8a11-ecfd84cf08c5 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:08.094990 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496e34f-3b5e-4cb3-ad3c-e4981a660f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:08.095710 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496e34f-3b5e-4cb3-ad3c-e4981a660f6b admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:08.096110 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496e34f-3b5e-4cb3-ad3c-e4981a660f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:08.096480 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496e34f-3b5e-4cb3-ad3c-e4981a660f6b admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:08.097103 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496e34f-3b5e-4cb3-ad3c-e4981a660f6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:08.099026 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496e34f-3b5e-4cb3-ad3c-e4981a660f6b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:08.108932 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1302/2584] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:30:08.119967 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e132480e-36a1-4354-9f05-981f284eed95 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:08.121352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e132480e-36a1-4354-9f05-981f284eed95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:08.122068 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1283/2585] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:30:08 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:30:08.126028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496e34f-3b5e-4cb3-ad3c-e4981a660f6b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:08.135349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496e34f-3b5e-4cb3-ad3c-e4981a660f6b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:08.139888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa41d9-6a2d-4cdf-b66b-6d4e50e35c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:08.140453 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa41d9-6a2d-4cdf-b66b-6d4e50e35c15 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:08.140727 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa41d9-6a2d-4cdf-b66b-6d4e50e35c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:08.140995 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa41d9-6a2d-4cdf-b66b-6d4e50e35c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:08.141620 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa41d9-6a2d-4cdf-b66b-6d4e50e35c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:08.143196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa41d9-6a2d-4cdf-b66b-6d4e50e35c15 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:08.145166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:08.158601 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1284/2586] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:30:08.159582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa41d9-6a2d-4cdf-b66b-6d4e50e35c15 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:08.170061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa41d9-6a2d-4cdf-b66b-6d4e50e35c15 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:08.179645 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0aeef-8dd9-4c1a-bd4f-50f1eb9c4151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:08.180269 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0aeef-8dd9-4c1a-bd4f-50f1eb9c4151 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:08.180549 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0aeef-8dd9-4c1a-bd4f-50f1eb9c4151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:08.180815 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0aeef-8dd9-4c1a-bd4f-50f1eb9c4151 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:08.181451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de0aeef-8dd9-4c1a-bd4f-50f1eb9c4151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:08.181579 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e132480e-36a1-4354-9f05-981f284eed95 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:08.183026 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de0aeef-8dd9-4c1a-bd4f-50f1eb9c4151 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:08.185306 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3201fe-07cb-45b5-acc6-7f729c1c7ea4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:08.185904 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3201fe-07cb-45b5-acc6-7f729c1c7ea4 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:08.186185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3201fe-07cb-45b5-acc6-7f729c1c7ea4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:08.186458 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3201fe-07cb-45b5-acc6-7f729c1c7ea4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:08.188246 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-0f3201fe-07cb-45b5-acc6-7f729c1c7ea4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:08.198015 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1285/2587] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:08.199214 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0aeef-8dd9-4c1a-bd4f-50f1eb9c4151 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:08.208790 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0aeef-8dd9-4c1a-bd4f-50f1eb9c4151 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:08.214084 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38400f3f-f5e1-45fa-be2b-aa96a7be075b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:08.214996 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38400f3f-f5e1-45fa-be2b-aa96a7be075b admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:08.215273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38400f3f-f5e1-45fa-be2b-aa96a7be075b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:08.216072 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38400f3f-f5e1-45fa-be2b-aa96a7be075b admin admin] PATH_INFO: `/v3/projects/268f6f847a9e4be2b6af38dbda2e7ba4/users/c4fef716c3e04ce9a55bb6ab2bdbd84d/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:08.231837 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38400f3f-f5e1-45fa-be2b-aa96a7be075b admin admin] RBAC: Authorizing `identity:create_grant(project_id=268f6f847a9e4be2b6af38dbda2e7ba4, user_id=c4fef716c3e04ce9a55bb6ab2bdbd84d, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:08.234472 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38400f3f-f5e1-45fa-be2b-aa96a7be075b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:08.247122 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1286/2588] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:08 2022] PUT /identity/v3/projects/268f6f847a9e4be2b6af38dbda2e7ba4/users/c4fef716c3e04ce9a55bb6ab2bdbd84d/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:30:08.252475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3201fe-07cb-45b5-acc6-7f729c1c7ea4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:08.253981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3201fe-07cb-45b5-acc6-7f729c1c7ea4 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:08.257297 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc60b8-caf8-4c78-bc10-0a7f30df5995 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:08.257580 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc60b8-caf8-4c78-bc10-0a7f30df5995 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:08.257848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc60b8-caf8-4c78-bc10-0a7f30df5995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:08.263642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1303/2589] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:30:08.285992 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-fccc60b8-caf8-4c78-bc10-0a7f30df5995 None None] MFA Rules not processed for user `c4fef716c3e04ce9a55bb6ab2bdbd84d`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:08.287196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-fccc60b8-caf8-4c78-bc10-0a7f30df5995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:08.350650 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1287/2590] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:30:08.355181 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3201fe-07cb-45b5-acc6-7f729c1c7ea4 service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:08.365811 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3201fe-07cb-45b5-acc6-7f729c1c7ea4 service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:08.426728 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258e13d-c155-43be-8f3f-58406fe22526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:08.427334 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258e13d-c155-43be-8f3f-58406fe22526 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:08.427615 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258e13d-c155-43be-8f3f-58406fe22526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:08.427930 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258e13d-c155-43be-8f3f-58406fe22526 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:08.428556 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258e13d-c155-43be-8f3f-58406fe22526 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:08.430531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258e13d-c155-43be-8f3f-58406fe22526 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:08.458330 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1304/2591] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:30:08 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:30:08.462952 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc60b8-caf8-4c78-bc10-0a7f30df5995 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:08.475362 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc60b8-caf8-4c78-bc10-0a7f30df5995 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:08.543278 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fed3f2-7c92-42c4-8b5e-43d90dc4f003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:08.544146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fed3f2-7c92-42c4-8b5e-43d90dc4f003 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:08.544525 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fed3f2-7c92-42c4-8b5e-43d90dc4f003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:08.544899 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fed3f2-7c92-42c4-8b5e-43d90dc4f003 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:08.545820 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fed3f2-7c92-42c4-8b5e-43d90dc4f003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:08.548312 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fed3f2-7c92-42c4-8b5e-43d90dc4f003 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:08.573130 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1288/2592] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:30:08 2022] POST /identity/v3/users => generated 298 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:30:08.577701 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258e13d-c155-43be-8f3f-58406fe22526 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:08.588416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258e13d-c155-43be-8f3f-58406fe22526 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:08.594258 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4087d24-1b11-4c1d-8830-0a58b848c762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:08.594877 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4087d24-1b11-4c1d-8830-0a58b848c762 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:08.595230 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4087d24-1b11-4c1d-8830-0a58b848c762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:08.595630 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4087d24-1b11-4c1d-8830-0a58b848c762 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:08.596424 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4087d24-1b11-4c1d-8830-0a58b848c762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:08.598016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4087d24-1b11-4c1d-8830-0a58b848c762 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:08.609554 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1305/2593] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:08.613810 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fed3f2-7c92-42c4-8b5e-43d90dc4f003 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:08.625058 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fed3f2-7c92-42c4-8b5e-43d90dc4f003 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:08.629961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c94c0c-ea1e-44e9-9ba5-3bbfdbb82642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:08.630605 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c94c0c-ea1e-44e9-9ba5-3bbfdbb82642 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:08.630892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c94c0c-ea1e-44e9-9ba5-3bbfdbb82642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:08.631164 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c94c0c-ea1e-44e9-9ba5-3bbfdbb82642 admin admin] PATH_INFO: `/v3/projects/afce72db755a4437afaee0e7d7493d39/users/9b45389f81014fd193fe59a954e47ca0/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:08.648494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c94c0c-ea1e-44e9-9ba5-3bbfdbb82642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afce72db755a4437afaee0e7d7493d39, user_id=9b45389f81014fd193fe59a954e47ca0, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:08.651169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c94c0c-ea1e-44e9-9ba5-3bbfdbb82642 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:08.665746 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1289/2594] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:08 2022] PUT /identity/v3/projects/afce72db755a4437afaee0e7d7493d39/users/9b45389f81014fd193fe59a954e47ca0/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:08.670061 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4087d24-1b11-4c1d-8830-0a58b848c762 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:08.680277 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4087d24-1b11-4c1d-8830-0a58b848c762 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:08.697152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:08.738118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca309dab-8dbb-4682-8492-6ca273cbd225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:08.738649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca309dab-8dbb-4682-8492-6ca273cbd225 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:08.738924 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca309dab-8dbb-4682-8492-6ca273cbd225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:08.739196 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca309dab-8dbb-4682-8492-6ca273cbd225 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:08.739857 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca309dab-8dbb-4682-8492-6ca273cbd225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:08.741445 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca309dab-8dbb-4682-8492-6ca273cbd225 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:08.751550 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1306/2595] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:08.755795 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c94c0c-ea1e-44e9-9ba5-3bbfdbb82642 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:08.767425 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c94c0c-ea1e-44e9-9ba5-3bbfdbb82642 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:08.772464 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cf0b6-4efd-433f-97ea-f55a7dc5449e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:08.773109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cf0b6-4efd-433f-97ea-f55a7dc5449e admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:08.773387 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cf0b6-4efd-433f-97ea-f55a7dc5449e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:08.773657 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cf0b6-4efd-433f-97ea-f55a7dc5449e admin admin] PATH_INFO: `/v3/projects/afce72db755a4437afaee0e7d7493d39/users/9b45389f81014fd193fe59a954e47ca0/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:08.775669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6cf0b6-4efd-433f-97ea-f55a7dc5449e admin admin] RBAC: Authorizing `identity:create_grant(project_id=afce72db755a4437afaee0e7d7493d39, user_id=9b45389f81014fd193fe59a954e47ca0, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:08.778337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6cf0b6-4efd-433f-97ea-f55a7dc5449e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:08.788705 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1290/2596] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:08 2022] PUT /identity/v3/projects/afce72db755a4437afaee0e7d7493d39/users/9b45389f81014fd193fe59a954e47ca0/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:30:08.800155 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc246e0-5ca9-468a-bb69-f485729852cf None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:08.800495 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc246e0-5ca9-468a-bb69-f485729852cf None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:08.800774 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc246e0-5ca9-468a-bb69-f485729852cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:08.829867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-7cc246e0-5ca9-468a-bb69-f485729852cf None None] MFA Rules not processed for user `9b45389f81014fd193fe59a954e47ca0`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:08.831108 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-7cc246e0-5ca9-468a-bb69-f485729852cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:08.899050 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1307/2597] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:08 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:08.910389 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cf0b6-4efd-433f-97ea-f55a7dc5449e admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:08.916052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:08.966901 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cf0b6-4efd-433f-97ea-f55a7dc5449e admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:08.970514 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbee72d-dfe0-4bb6-8033-5eba5e115133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:08.971106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbee72d-dfe0-4bb6-8033-5eba5e115133 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:08.971383 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbee72d-dfe0-4bb6-8033-5eba5e115133 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:08.971651 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbee72d-dfe0-4bb6-8033-5eba5e115133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:09.026043 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbee72d-dfe0-4bb6-8033-5eba5e115133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:09.027394 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbee72d-dfe0-4bb6-8033-5eba5e115133 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:09.035973 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1291/2598] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:12.884995 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc246e0-5ca9-468a-bb69-f485729852cf None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:12.886414 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-7cc246e0-5ca9-468a-bb69-f485729852cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:12.899105 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc246e0-5ca9-468a-bb69-f485729852cf None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:12.903371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dddf48-394e-46a4-bf5d-c6dd3fc32b69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:12.904026 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dddf48-394e-46a4-bf5d-c6dd3fc32b69 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:12.904303 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dddf48-394e-46a4-bf5d-c6dd3fc32b69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:12.904568 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dddf48-394e-46a4-bf5d-c6dd3fc32b69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:13.089515 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dddf48-394e-46a4-bf5d-c6dd3fc32b69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:13.091485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dddf48-394e-46a4-bf5d-c6dd3fc32b69 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:13.100960 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1308/2599] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:17.459995 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbee72d-dfe0-4bb6-8033-5eba5e115133 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:17.469603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbee72d-dfe0-4bb6-8033-5eba5e115133 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:17.473633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ea94a-811c-4bb8-b49d-c15804237a40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:17.474257 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ea94a-811c-4bb8-b49d-c15804237a40 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:17.474537 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ea94a-811c-4bb8-b49d-c15804237a40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:17.474809 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ea94a-811c-4bb8-b49d-c15804237a40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:17.484197 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ea94a-811c-4bb8-b49d-c15804237a40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:17.485645 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ea94a-811c-4bb8-b49d-c15804237a40 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:17.495804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1292/2600] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:17 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:23.096805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=84 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:24.537482 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dddf48-394e-46a4-bf5d-c6dd3fc32b69 service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:24.602271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dddf48-394e-46a4-bf5d-c6dd3fc32b69 service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:24.609884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29182d74-1401-4843-8d8c-bbcf03aaca82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:24.609884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29182d74-1401-4843-8d8c-bbcf03aaca82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:24.609884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29182d74-1401-4843-8d8c-bbcf03aaca82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:24.609884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29182d74-1401-4843-8d8c-bbcf03aaca82 admin admin] PATH_INFO: `/v3/users/c4fef716c3e04ce9a55bb6ab2bdbd84d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:24.610363 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29182d74-1401-4843-8d8c-bbcf03aaca82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4fef716c3e04ce9a55bb6ab2bdbd84d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:24.612306 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29182d74-1401-4843-8d8c-bbcf03aaca82 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:24.663361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-29182d74-1401-4843-8d8c-bbcf03aaca82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4fef716c3e04ce9a55bb6ab2bdbd84d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:24.664081 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-29182d74-1401-4843-8d8c-bbcf03aaca82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4fef716c3e04ce9a55bb6ab2bdbd84d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:24.664705 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-29182d74-1401-4843-8d8c-bbcf03aaca82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4fef716c3e04ce9a55bb6ab2bdbd84d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:24.676562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-29182d74-1401-4843-8d8c-bbcf03aaca82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4fef716c3e04ce9a55bb6ab2bdbd84d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:24.688697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-29182d74-1401-4843-8d8c-bbcf03aaca82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4fef716c3e04ce9a55bb6ab2bdbd84d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:24.702807 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1309/2601] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:30:24 2022] DELETE /identity/v3/users/c4fef716c3e04ce9a55bb6ab2bdbd84d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:24.707639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778d7ec-c133-4843-b637-28a944e41564 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:24.708183 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778d7ec-c133-4843-b637-28a944e41564 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:24.708302 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778d7ec-c133-4843-b637-28a944e41564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:24.730853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-0778d7ec-c133-4843-b637-28a944e41564 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:24.732740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0778d7ec-c133-4843-b637-28a944e41564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:24.802455 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1293/2602] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:24.812815 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29182d74-1401-4843-8d8c-bbcf03aaca82 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:24.814212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-29182d74-1401-4843-8d8c-bbcf03aaca82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:24.872683 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29182d74-1401-4843-8d8c-bbcf03aaca82 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:24.876540 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2207c-5b27-4ec7-9dc3-a77e6ab32c7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:24.877133 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2207c-5b27-4ec7-9dc3-a77e6ab32c7d service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:24.877405 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2207c-5b27-4ec7-9dc3-a77e6ab32c7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:24.877670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2207c-5b27-4ec7-9dc3-a77e6ab32c7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:24.886717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2207c-5b27-4ec7-9dc3-a77e6ab32c7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:24.888804 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2207c-5b27-4ec7-9dc3-a77e6ab32c7d service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:24.898252 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1310/2603] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:25.162601 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778d7ec-c133-4843-b637-28a944e41564 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:25.164142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-0778d7ec-c133-4843-b637-28a944e41564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:25.179317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778d7ec-c133-4843-b637-28a944e41564 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:25.184138 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54408c7-325e-4d5c-bf9a-21c197bb1ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:25.185273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54408c7-325e-4d5c-bf9a-21c197bb1ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:25.185579 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54408c7-325e-4d5c-bf9a-21c197bb1ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:25.185809 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54408c7-325e-4d5c-bf9a-21c197bb1ab8 admin admin] PATH_INFO: `/v3/projects/268f6f847a9e4be2b6af38dbda2e7ba4` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:25.187105 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54408c7-325e-4d5c-bf9a-21c197bb1ab8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=268f6f847a9e4be2b6af38dbda2e7ba4)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:25.189061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54408c7-325e-4d5c-bf9a-21c197bb1ab8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:25.229964 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:25.237664 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c54408c7-325e-4d5c-bf9a-21c197bb1ab8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '268f6f847a9e4be2b6af38dbda2e7ba4', 'request_id': 'req-c54408c7-325e-4d5c-bf9a-21c197bb1ab8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:25.244016 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c54408c7-325e-4d5c-bf9a-21c197bb1ab8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '268f6f847a9e4be2b6af38dbda2e7ba4', 'request_id': 'req-c54408c7-325e-4d5c-bf9a-21c197bb1ab8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:25.250308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c54408c7-325e-4d5c-bf9a-21c197bb1ab8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '268f6f847a9e4be2b6af38dbda2e7ba4', 'request_id': 'req-c54408c7-325e-4d5c-bf9a-21c197bb1ab8'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:25.262202 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c54408c7-325e-4d5c-bf9a-21c197bb1ab8 admin admin] The token cache is being invalidate because project 268f6f847a9e4be2b6af38dbda2e7ba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:30:25.262489 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c54408c7-325e-4d5c-bf9a-21c197bb1ab8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:25.263115 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c54408c7-325e-4d5c-bf9a-21c197bb1ab8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:25.264626 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1294/2604] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:30:25 2022] DELETE /identity/v3/projects/268f6f847a9e4be2b6af38dbda2e7ba4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:25.268587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2207c-5b27-4ec7-9dc3-a77e6ab32c7d service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:25.278099 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2207c-5b27-4ec7-9dc3-a77e6ab32c7d service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:25.297088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:25.336292 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c2df0-da59-46ac-9fed-35d92aa04a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:25.336910 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c2df0-da59-46ac-9fed-35d92aa04a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:25.337191 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c2df0-da59-46ac-9fed-35d92aa04a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:25.337461 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c2df0-da59-46ac-9fed-35d92aa04a49 admin admin] PATH_INFO: `/v3/users/9b45389f81014fd193fe59a954e47ca0` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:25.338574 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c2df0-da59-46ac-9fed-35d92aa04a49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b45389f81014fd193fe59a954e47ca0)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:25.340553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c2df0-da59-46ac-9fed-35d92aa04a49 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:25.385848 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-448c2df0-da59-46ac-9fed-35d92aa04a49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b45389f81014fd193fe59a954e47ca0'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:25.386487 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-448c2df0-da59-46ac-9fed-35d92aa04a49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b45389f81014fd193fe59a954e47ca0'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:25.387083 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-448c2df0-da59-46ac-9fed-35d92aa04a49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b45389f81014fd193fe59a954e47ca0'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:25.394927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-448c2df0-da59-46ac-9fed-35d92aa04a49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b45389f81014fd193fe59a954e47ca0'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:25.407532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-448c2df0-da59-46ac-9fed-35d92aa04a49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b45389f81014fd193fe59a954e47ca0'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:25.421107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1311/2605] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:30:25 2022] DELETE /identity/v3/users/9b45389f81014fd193fe59a954e47ca0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:25.690960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54408c7-325e-4d5c-bf9a-21c197bb1ab8 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:25.692471 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c54408c7-325e-4d5c-bf9a-21c197bb1ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:25.705909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54408c7-325e-4d5c-bf9a-21c197bb1ab8 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:25.763693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f604bd49-0673-443e-9d18-fac6839da3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:25.764383 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604bd49-0673-443e-9d18-fac6839da3a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:25.764643 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604bd49-0673-443e-9d18-fac6839da3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:25.764914 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604bd49-0673-443e-9d18-fac6839da3a3 admin admin] PATH_INFO: `/v3/projects/afce72db755a4437afaee0e7d7493d39` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:25.766023 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f604bd49-0673-443e-9d18-fac6839da3a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afce72db755a4437afaee0e7d7493d39)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:25.767959 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f604bd49-0673-443e-9d18-fac6839da3a3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:25.813607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f604bd49-0673-443e-9d18-fac6839da3a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afce72db755a4437afaee0e7d7493d39', 'request_id': 'req-f604bd49-0673-443e-9d18-fac6839da3a3'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:25.820901 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f604bd49-0673-443e-9d18-fac6839da3a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afce72db755a4437afaee0e7d7493d39', 'request_id': 'req-f604bd49-0673-443e-9d18-fac6839da3a3'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:25.828587 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f604bd49-0673-443e-9d18-fac6839da3a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afce72db755a4437afaee0e7d7493d39', 'request_id': 'req-f604bd49-0673-443e-9d18-fac6839da3a3'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:25.845399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f604bd49-0673-443e-9d18-fac6839da3a3 admin admin] The token cache is being invalidate because project afce72db755a4437afaee0e7d7493d39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:30:25.845690 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f604bd49-0673-443e-9d18-fac6839da3a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:25.846341 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f604bd49-0673-443e-9d18-fac6839da3a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:25.847891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1295/2606] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:30:25 2022] DELETE /identity/v3/projects/afce72db755a4437afaee0e7d7493d39 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:25.853773 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a3f86-e43e-4946-9fdf-893ee6598c46 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:25.854267 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a3f86-e43e-4946-9fdf-893ee6598c46 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:25.854368 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a3f86-e43e-4946-9fdf-893ee6598c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:25.875228 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-2c4a3f86-e43e-4946-9fdf-893ee6598c46 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:25.876462 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2c4a3f86-e43e-4946-9fdf-893ee6598c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:25.933476 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1312/2607] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:25.941758 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92658108-4a7d-4247-9daa-9d1e644eeea4 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:25.942078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92658108-4a7d-4247-9daa-9d1e644eeea4 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:25.942352 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92658108-4a7d-4247-9daa-9d1e644eeea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:25.962765 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-92658108-4a7d-4247-9daa-9d1e644eeea4 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:25.964004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-92658108-4a7d-4247-9daa-9d1e644eeea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:25.972647 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1296/2608] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:25.976881 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a3f86-e43e-4946-9fdf-893ee6598c46 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:25.986646 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a3f86-e43e-4946-9fdf-893ee6598c46 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:25.991167 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70f30f-a751-434f-a42c-9d5dbd327654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:25.991752 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70f30f-a751-434f-a42c-9d5dbd327654 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:25.992063 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70f30f-a751-434f-a42c-9d5dbd327654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:25.992336 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70f30f-a751-434f-a42c-9d5dbd327654 admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:25.992972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70f30f-a751-434f-a42c-9d5dbd327654 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:30:25.993298 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70f30f-a751-434f-a42c-9d5dbd327654 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:25.994863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70f30f-a751-434f-a42c-9d5dbd327654 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:26.006160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1313/2609] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:30:26.010427 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92658108-4a7d-4247-9daa-9d1e644eeea4 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:26.020400 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92658108-4a7d-4247-9daa-9d1e644eeea4 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:26.025215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208fdbb-3959-4d55-a142-9ed6d49d3462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:26.025800 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208fdbb-3959-4d55-a142-9ed6d49d3462 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:26.026078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208fdbb-3959-4d55-a142-9ed6d49d3462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:26.026346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208fdbb-3959-4d55-a142-9ed6d49d3462 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:26.026992 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208fdbb-3959-4d55-a142-9ed6d49d3462 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:26.029547 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208fdbb-3959-4d55-a142-9ed6d49d3462 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:26.056848 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1297/2610] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:30:26 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:30:26.060839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70f30f-a751-434f-a42c-9d5dbd327654 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:26.070945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70f30f-a751-434f-a42c-9d5dbd327654 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:26.096808 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:26.129409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb93455-c31b-4e2a-9139-b9aa1d2634fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:26.130004 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb93455-c31b-4e2a-9139-b9aa1d2634fa admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:26.130282 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb93455-c31b-4e2a-9139-b9aa1d2634fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:26.130550 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb93455-c31b-4e2a-9139-b9aa1d2634fa admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:26.131178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb93455-c31b-4e2a-9139-b9aa1d2634fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:26.133090 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb93455-c31b-4e2a-9139-b9aa1d2634fa admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:26.153421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1314/2611] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:30:26 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:30:26.157409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208fdbb-3959-4d55-a142-9ed6d49d3462 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:26.167654 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208fdbb-3959-4d55-a142-9ed6d49d3462 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:26.172777 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2711b4b-b702-4fcf-934b-7b1eb3338b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:26.173362 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2711b4b-b702-4fcf-934b-7b1eb3338b7b admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:26.173639 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2711b4b-b702-4fcf-934b-7b1eb3338b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:26.173912 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2711b4b-b702-4fcf-934b-7b1eb3338b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:26.174558 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2711b4b-b702-4fcf-934b-7b1eb3338b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:26.176171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2711b4b-b702-4fcf-934b-7b1eb3338b7b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:26.186446 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1298/2612] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:26.190215 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb93455-c31b-4e2a-9139-b9aa1d2634fa admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:26.200294 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb93455-c31b-4e2a-9139-b9aa1d2634fa admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:26.205267 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce4f10-a4bb-4d07-842b-3f49d4353172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:26.205854 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce4f10-a4bb-4d07-842b-3f49d4353172 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:26.206132 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce4f10-a4bb-4d07-842b-3f49d4353172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:26.206404 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce4f10-a4bb-4d07-842b-3f49d4353172 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:26.207038 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce4f10-a4bb-4d07-842b-3f49d4353172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:26.208636 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce4f10-a4bb-4d07-842b-3f49d4353172 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:26.218371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1315/2613] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:26.222266 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2711b4b-b702-4fcf-934b-7b1eb3338b7b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:26.233011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2711b4b-b702-4fcf-934b-7b1eb3338b7b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:26.242756 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a4b1b-0d6d-4964-ab55-f67235f647d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:26.243417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a4b1b-0d6d-4964-ab55-f67235f647d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:26.243700 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a4b1b-0d6d-4964-ab55-f67235f647d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:26.244001 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a4b1b-0d6d-4964-ab55-f67235f647d9 admin admin] PATH_INFO: `/v3/projects/a93679e739a44998bdfe5a832eef1e37/users/a2e67c453dac40c9bdbf3b77d47c2cc1/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:26.259206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a4b1b-0d6d-4964-ab55-f67235f647d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a93679e739a44998bdfe5a832eef1e37, user_id=a2e67c453dac40c9bdbf3b77d47c2cc1, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:26.261881 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a4b1b-0d6d-4964-ab55-f67235f647d9 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:26.265163 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:26.276825 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1299/2614] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:26 2022] PUT /identity/v3/projects/a93679e739a44998bdfe5a832eef1e37/users/a2e67c453dac40c9bdbf3b77d47c2cc1/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:26.289000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3921c-e2ca-4a4e-8641-dc18091e7965 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:26.289289 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3921c-e2ca-4a4e-8641-dc18091e7965 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:26.289563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3921c-e2ca-4a4e-8641-dc18091e7965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:26.318971 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-88a3921c-e2ca-4a4e-8641-dc18091e7965 None None] MFA Rules not processed for user `a2e67c453dac40c9bdbf3b77d47c2cc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:26.320218 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-88a3921c-e2ca-4a4e-8641-dc18091e7965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:26.384001 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1316/2615] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:26.396399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a4b1b-0d6d-4964-ab55-f67235f647d9 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:26.398364 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f47a4b1b-0d6d-4964-ab55-f67235f647d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:26.463433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a4b1b-0d6d-4964-ab55-f67235f647d9 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:26.467610 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971ba46-cf78-44db-a112-997ddeb6f8bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:26.468244 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971ba46-cf78-44db-a112-997ddeb6f8bc service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:26.468525 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971ba46-cf78-44db-a112-997ddeb6f8bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:26.468797 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971ba46-cf78-44db-a112-997ddeb6f8bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:26.478330 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2971ba46-cf78-44db-a112-997ddeb6f8bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:26.479857 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2971ba46-cf78-44db-a112-997ddeb6f8bc service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:26.489233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1300/2616] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:26.511024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3921c-e2ca-4a4e-8641-dc18091e7965 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:26.573384 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3921c-e2ca-4a4e-8641-dc18091e7965 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:26.578545 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee369931-d20e-4e8c-b9f6-e9e559d24f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:26.579172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee369931-d20e-4e8c-b9f6-e9e559d24f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:26.579448 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee369931-d20e-4e8c-b9f6-e9e559d24f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:26.579718 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee369931-d20e-4e8c-b9f6-e9e559d24f41 admin admin] PATH_INFO: `/v3/users/a2e67c453dac40c9bdbf3b77d47c2cc1` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:26.580914 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee369931-d20e-4e8c-b9f6-e9e559d24f41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2e67c453dac40c9bdbf3b77d47c2cc1)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:26.582864 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee369931-d20e-4e8c-b9f6-e9e559d24f41 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:26.622978 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:26.630291 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ee369931-d20e-4e8c-b9f6-e9e559d24f41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2e67c453dac40c9bdbf3b77d47c2cc1'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:26.631020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ee369931-d20e-4e8c-b9f6-e9e559d24f41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2e67c453dac40c9bdbf3b77d47c2cc1'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:26.631809 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ee369931-d20e-4e8c-b9f6-e9e559d24f41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2e67c453dac40c9bdbf3b77d47c2cc1'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:26.639006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ee369931-d20e-4e8c-b9f6-e9e559d24f41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2e67c453dac40c9bdbf3b77d47c2cc1'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:26.650646 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ee369931-d20e-4e8c-b9f6-e9e559d24f41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2e67c453dac40c9bdbf3b77d47c2cc1'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:26.664643 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1317/2617] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:30:26 2022] DELETE /identity/v3/users/a2e67c453dac40c9bdbf3b77d47c2cc1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:26.670053 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9836276-db63-4933-9314-4b546142ab47 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:26.670348 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9836276-db63-4933-9314-4b546142ab47 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:26.670620 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9836276-db63-4933-9314-4b546142ab47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:26.689734 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e9836276-db63-4933-9314-4b546142ab47 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:26.690938 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e9836276-db63-4933-9314-4b546142ab47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:26.748506 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1301/2618] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:26.758291 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee369931-d20e-4e8c-b9f6-e9e559d24f41 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:26.759682 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ee369931-d20e-4e8c-b9f6-e9e559d24f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:26.823668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee369931-d20e-4e8c-b9f6-e9e559d24f41 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:26.829294 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f778bfe-369f-4219-9f80-c259d67d165e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:26.829890 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f778bfe-369f-4219-9f80-c259d67d165e service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:26.830169 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f778bfe-369f-4219-9f80-c259d67d165e service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:26.830832 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f778bfe-369f-4219-9f80-c259d67d165e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:26.842872 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f778bfe-369f-4219-9f80-c259d67d165e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:26.844397 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f778bfe-369f-4219-9f80-c259d67d165e service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:26.854074 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1318/2619] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:27.117241 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9836276-db63-4933-9314-4b546142ab47 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:27.119094 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e9836276-db63-4933-9314-4b546142ab47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:27.132316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9836276-db63-4933-9314-4b546142ab47 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:27.141232 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad130d0-b270-4bc0-aebb-e19d831ca6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:27.141966 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad130d0-b270-4bc0-aebb-e19d831ca6dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:27.142314 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad130d0-b270-4bc0-aebb-e19d831ca6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:27.142652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad130d0-b270-4bc0-aebb-e19d831ca6dd admin admin] PATH_INFO: `/v3/projects/a93679e739a44998bdfe5a832eef1e37` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:27.143970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad130d0-b270-4bc0-aebb-e19d831ca6dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a93679e739a44998bdfe5a832eef1e37)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:27.146149 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad130d0-b270-4bc0-aebb-e19d831ca6dd admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:27.196430 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-aad130d0-b270-4bc0-aebb-e19d831ca6dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a93679e739a44998bdfe5a832eef1e37', 'request_id': 'req-aad130d0-b270-4bc0-aebb-e19d831ca6dd'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:27.204810 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-aad130d0-b270-4bc0-aebb-e19d831ca6dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a93679e739a44998bdfe5a832eef1e37', 'request_id': 'req-aad130d0-b270-4bc0-aebb-e19d831ca6dd'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:27.211285 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-aad130d0-b270-4bc0-aebb-e19d831ca6dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a93679e739a44998bdfe5a832eef1e37', 'request_id': 'req-aad130d0-b270-4bc0-aebb-e19d831ca6dd'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:27.224025 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-aad130d0-b270-4bc0-aebb-e19d831ca6dd admin admin] The token cache is being invalidate because project a93679e739a44998bdfe5a832eef1e37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:30:27.224323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-aad130d0-b270-4bc0-aebb-e19d831ca6dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:27.224972 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-aad130d0-b270-4bc0-aebb-e19d831ca6dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:27.226599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1302/2620] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:30:27 2022] DELETE /identity/v3/projects/a93679e739a44998bdfe5a832eef1e37 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:27.232598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bfdb4-fd05-488b-939d-6fbd94032a94 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:27.232881 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bfdb4-fd05-488b-939d-6fbd94032a94 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:27.233153 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bfdb4-fd05-488b-939d-6fbd94032a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:27.253150 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-084bfdb4-fd05-488b-939d-6fbd94032a94 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:27.254353 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-084bfdb4-fd05-488b-939d-6fbd94032a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:27.311942 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1319/2621] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:27.319562 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc62bd1-abee-4d6f-a4fe-227adacb57e8 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:27.319882 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc62bd1-abee-4d6f-a4fe-227adacb57e8 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:27.320163 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc62bd1-abee-4d6f-a4fe-227adacb57e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:27.340772 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-afc62bd1-abee-4d6f-a4fe-227adacb57e8 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:27.341975 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-afc62bd1-abee-4d6f-a4fe-227adacb57e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:27.349818 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1303/2622] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:27.353986 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bfdb4-fd05-488b-939d-6fbd94032a94 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:27.363590 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bfdb4-fd05-488b-939d-6fbd94032a94 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:27.368184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383e43c-37e8-4b84-93cd-7fdc9f42756a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:27.368761 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383e43c-37e8-4b84-93cd-7fdc9f42756a admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:27.369035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383e43c-37e8-4b84-93cd-7fdc9f42756a admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:27.369300 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383e43c-37e8-4b84-93cd-7fdc9f42756a admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:27.369927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383e43c-37e8-4b84-93cd-7fdc9f42756a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:30:27.370223 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383e43c-37e8-4b84-93cd-7fdc9f42756a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:27.372360 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383e43c-37e8-4b84-93cd-7fdc9f42756a admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:27.383362 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1320/2623] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:30:27.388093 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc62bd1-abee-4d6f-a4fe-227adacb57e8 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:27.397375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc62bd1-abee-4d6f-a4fe-227adacb57e8 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:27.402233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cb2cd-ff79-4ed6-b457-3da06fe6e6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:27.402821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cb2cd-ff79-4ed6-b457-3da06fe6e6e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:27.403093 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cb2cd-ff79-4ed6-b457-3da06fe6e6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:27.403359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cb2cd-ff79-4ed6-b457-3da06fe6e6e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:27.404027 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009cb2cd-ff79-4ed6-b457-3da06fe6e6e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:27.405963 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009cb2cd-ff79-4ed6-b457-3da06fe6e6e6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:27.420885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:27.433432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1304/2624] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:30:27 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:30:27.438181 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383e43c-37e8-4b84-93cd-7fdc9f42756a admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:27.448035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383e43c-37e8-4b84-93cd-7fdc9f42756a admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:27.490653 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:27.505617 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194bcc8a-34ab-453f-961d-2d5e3b5fc9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:27.507259 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bcc8a-34ab-453f-961d-2d5e3b5fc9f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:27.507259 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bcc8a-34ab-453f-961d-2d5e3b5fc9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:27.507259 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bcc8a-34ab-453f-961d-2d5e3b5fc9f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:27.507570 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194bcc8a-34ab-453f-961d-2d5e3b5fc9f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:27.509328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194bcc8a-34ab-453f-961d-2d5e3b5fc9f9 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:27.529982 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1321/2625] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:30:27 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:30:27.534068 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cb2cd-ff79-4ed6-b457-3da06fe6e6e6 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:27.544886 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cb2cd-ff79-4ed6-b457-3da06fe6e6e6 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:27.549563 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d1a61-0b1a-4136-963b-8d1c67971d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:27.550151 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d1a61-0b1a-4136-963b-8d1c67971d0b admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:27.550426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d1a61-0b1a-4136-963b-8d1c67971d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:27.550702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d1a61-0b1a-4136-963b-8d1c67971d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:27.551363 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d1a61-0b1a-4136-963b-8d1c67971d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:27.552977 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d1a61-0b1a-4136-963b-8d1c67971d0b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:27.563392 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1305/2626] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:30:27.567297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194bcc8a-34ab-453f-961d-2d5e3b5fc9f9 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:27.577327 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194bcc8a-34ab-453f-961d-2d5e3b5fc9f9 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:27.581863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1d505-6a16-4368-b671-4d419ffeefc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:27.582454 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1d505-6a16-4368-b671-4d419ffeefc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:27.582728 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1d505-6a16-4368-b671-4d419ffeefc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:27.582996 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1d505-6a16-4368-b671-4d419ffeefc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:27.583622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c1d505-6a16-4368-b671-4d419ffeefc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:27.585288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c1d505-6a16-4368-b671-4d419ffeefc7 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:27.595546 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1322/2627] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:27.599479 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d1a61-0b1a-4136-963b-8d1c67971d0b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:27.609802 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d1a61-0b1a-4136-963b-8d1c67971d0b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:27.614360 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6211a4-ad62-4d50-8376-5d4c10155c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:27.614984 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6211a4-ad62-4d50-8376-5d4c10155c05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:27.615258 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6211a4-ad62-4d50-8376-5d4c10155c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:27.615526 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6211a4-ad62-4d50-8376-5d4c10155c05 admin admin] PATH_INFO: `/v3/projects/f446bc44fa074e80b2b477bdf13bcdcf/users/7dbae40715f9423b82b1ee34ef30f5ec/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:27.631349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6211a4-ad62-4d50-8376-5d4c10155c05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f446bc44fa074e80b2b477bdf13bcdcf, user_id=7dbae40715f9423b82b1ee34ef30f5ec, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:27.634332 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6211a4-ad62-4d50-8376-5d4c10155c05 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:27.646588 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1306/2628] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:27 2022] PUT /identity/v3/projects/f446bc44fa074e80b2b477bdf13bcdcf/users/7dbae40715f9423b82b1ee34ef30f5ec/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:27.657060 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f6d54-7661-44f3-b3ba-9dba0fe369e7 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:27.657344 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f6d54-7661-44f3-b3ba-9dba0fe369e7 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:27.657612 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f6d54-7661-44f3-b3ba-9dba0fe369e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:27.686499 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-406f6d54-7661-44f3-b3ba-9dba0fe369e7 None None] MFA Rules not processed for user `7dbae40715f9423b82b1ee34ef30f5ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:27.687697 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-406f6d54-7661-44f3-b3ba-9dba0fe369e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:27.750657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1323/2629] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:27.761475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6211a4-ad62-4d50-8376-5d4c10155c05 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:27.763152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bc6211a4-ad62-4d50-8376-5d4c10155c05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:27.818827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6211a4-ad62-4d50-8376-5d4c10155c05 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:27.822425 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea72efd-1075-4c3f-8f05-a28c898f7a99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:27.823015 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea72efd-1075-4c3f-8f05-a28c898f7a99 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:27.823292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea72efd-1075-4c3f-8f05-a28c898f7a99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:27.823563 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea72efd-1075-4c3f-8f05-a28c898f7a99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:27.832805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea72efd-1075-4c3f-8f05-a28c898f7a99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:27.834223 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea72efd-1075-4c3f-8f05-a28c898f7a99 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:27.844203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1307/2630] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:29.287409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec416b0-8021-48ee-9a92-cbd9315c09fa None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:29.288032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec416b0-8021-48ee-9a92-cbd9315c09fa None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:29.288677 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec416b0-8021-48ee-9a92-cbd9315c09fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:29.313765 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-5ec416b0-8021-48ee-9a92-cbd9315c09fa None None] MFA Rules not processed for user `7dbae40715f9423b82b1ee34ef30f5ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:29.315122 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5ec416b0-8021-48ee-9a92-cbd9315c09fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:29.325160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1324/2631] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:37.402040 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea72efd-1075-4c3f-8f05-a28c898f7a99 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:37.403546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8ea72efd-1075-4c3f-8f05-a28c898f7a99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:37.473394 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea72efd-1075-4c3f-8f05-a28c898f7a99 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:37.478069 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30f943-0502-436f-a7a5-4ff5f2b89ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:37.478697 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30f943-0502-436f-a7a5-4ff5f2b89ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:37.478977 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30f943-0502-436f-a7a5-4ff5f2b89ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:37.479249 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30f943-0502-436f-a7a5-4ff5f2b89ac2 admin admin] PATH_INFO: `/v3/users/3d5b88cce500405b8a6041ff44fee4ac` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:37.480413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30f943-0502-436f-a7a5-4ff5f2b89ac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d5b88cce500405b8a6041ff44fee4ac)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:37.482883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30f943-0502-436f-a7a5-4ff5f2b89ac2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:37.531826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ae30f943-0502-436f-a7a5-4ff5f2b89ac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d5b88cce500405b8a6041ff44fee4ac'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:37.539652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ae30f943-0502-436f-a7a5-4ff5f2b89ac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d5b88cce500405b8a6041ff44fee4ac'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:37.540380 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ae30f943-0502-436f-a7a5-4ff5f2b89ac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d5b88cce500405b8a6041ff44fee4ac'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:37.541037 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ae30f943-0502-436f-a7a5-4ff5f2b89ac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d5b88cce500405b8a6041ff44fee4ac'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:37.549238 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:37.555558 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ae30f943-0502-436f-a7a5-4ff5f2b89ac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d5b88cce500405b8a6041ff44fee4ac'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:37.569698 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1308/2632] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:30:37 2022] DELETE /identity/v3/users/3d5b88cce500405b8a6041ff44fee4ac => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:37.574276 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bf415-a558-420e-94e8-96f2cc17a3ad None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:37.574799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bf415-a558-420e-94e8-96f2cc17a3ad None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:37.575245 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bf415-a558-420e-94e8-96f2cc17a3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:37.598763 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-678bf415-a558-420e-94e8-96f2cc17a3ad None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:37.600080 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-678bf415-a558-420e-94e8-96f2cc17a3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:37.665336 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1325/2633] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:37.675818 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30f943-0502-436f-a7a5-4ff5f2b89ac2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:37.677236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ae30f943-0502-436f-a7a5-4ff5f2b89ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:37.740276 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30f943-0502-436f-a7a5-4ff5f2b89ac2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:37.744007 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a14766-0ff7-44c5-8fef-d65acb16db41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:37.744607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a14766-0ff7-44c5-8fef-d65acb16db41 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:37.744884 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a14766-0ff7-44c5-8fef-d65acb16db41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:37.745144 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a14766-0ff7-44c5-8fef-d65acb16db41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:37.754768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a14766-0ff7-44c5-8fef-d65acb16db41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:37.756232 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a14766-0ff7-44c5-8fef-d65acb16db41 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:37.765903 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1309/2634] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:38.038375 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678bf415-a558-420e-94e8-96f2cc17a3ad None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:38.039995 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-678bf415-a558-420e-94e8-96f2cc17a3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:38.054343 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678bf415-a558-420e-94e8-96f2cc17a3ad None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:38.061838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e19acb-f82c-403f-a4ac-1630f8718d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:38.064368 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e19acb-f82c-403f-a4ac-1630f8718d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:38.064680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e19acb-f82c-403f-a4ac-1630f8718d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:38.064955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e19acb-f82c-403f-a4ac-1630f8718d56 admin admin] PATH_INFO: `/v3/projects/89daac7d8a974f2783682b14b92920ac` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:38.066611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e19acb-f82c-403f-a4ac-1630f8718d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89daac7d8a974f2783682b14b92920ac)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:38.068649 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e19acb-f82c-403f-a4ac-1630f8718d56 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:38.119178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:38.125253 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a0e19acb-f82c-403f-a4ac-1630f8718d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89daac7d8a974f2783682b14b92920ac', 'request_id': 'req-a0e19acb-f82c-403f-a4ac-1630f8718d56'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:38.135911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a0e19acb-f82c-403f-a4ac-1630f8718d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89daac7d8a974f2783682b14b92920ac', 'request_id': 'req-a0e19acb-f82c-403f-a4ac-1630f8718d56'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:38.142631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a0e19acb-f82c-403f-a4ac-1630f8718d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89daac7d8a974f2783682b14b92920ac', 'request_id': 'req-a0e19acb-f82c-403f-a4ac-1630f8718d56'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:38.156641 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a0e19acb-f82c-403f-a4ac-1630f8718d56 admin admin] The token cache is being invalidate because project 89daac7d8a974f2783682b14b92920ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:30:38.156986 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a0e19acb-f82c-403f-a4ac-1630f8718d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:38.157612 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-a0e19acb-f82c-403f-a4ac-1630f8718d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:38.160139 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1326/2635] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:30:38 2022] DELETE /identity/v3/projects/89daac7d8a974f2783682b14b92920ac => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:38.164387 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a14766-0ff7-44c5-8fef-d65acb16db41 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:38.175977 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a14766-0ff7-44c5-8fef-d65acb16db41 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:38.241508 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc0e05-3da2-4866-bec9-08863b224e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:38.242204 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc0e05-3da2-4866-bec9-08863b224e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:38.242488 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc0e05-3da2-4866-bec9-08863b224e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:38.242768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc0e05-3da2-4866-bec9-08863b224e73 admin admin] PATH_INFO: `/v3/users/d9a46846eb064c73829e965dab71b88a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:38.244105 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc0e05-3da2-4866-bec9-08863b224e73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9a46846eb064c73829e965dab71b88a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:38.246041 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc0e05-3da2-4866-bec9-08863b224e73 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:38.319668 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6ebc0e05-3da2-4866-bec9-08863b224e73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9a46846eb064c73829e965dab71b88a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:38.328840 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6ebc0e05-3da2-4866-bec9-08863b224e73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9a46846eb064c73829e965dab71b88a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:38.329617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6ebc0e05-3da2-4866-bec9-08863b224e73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9a46846eb064c73829e965dab71b88a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:38.330529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6ebc0e05-3da2-4866-bec9-08863b224e73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9a46846eb064c73829e965dab71b88a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:38.345576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-6ebc0e05-3da2-4866-bec9-08863b224e73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9a46846eb064c73829e965dab71b88a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:38.363839 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1310/2636] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:30:38 2022] DELETE /identity/v3/users/d9a46846eb064c73829e965dab71b88a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:30:38.684221 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e19acb-f82c-403f-a4ac-1630f8718d56 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:38.685609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a0e19acb-f82c-403f-a4ac-1630f8718d56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:38.702908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e19acb-f82c-403f-a4ac-1630f8718d56 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:38.766691 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c87fd-2a46-4e35-ba68-33d317f9ea9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:38.767326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c87fd-2a46-4e35-ba68-33d317f9ea9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:38.767607 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c87fd-2a46-4e35-ba68-33d317f9ea9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:38.767920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c87fd-2a46-4e35-ba68-33d317f9ea9f admin admin] PATH_INFO: `/v3/projects/ea46ef8ffbdd41ce9192ce9cb80ddef6` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:38.769179 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c87fd-2a46-4e35-ba68-33d317f9ea9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea46ef8ffbdd41ce9192ce9cb80ddef6)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:38.772058 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c87fd-2a46-4e35-ba68-33d317f9ea9f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:38.819304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-953c87fd-2a46-4e35-ba68-33d317f9ea9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea46ef8ffbdd41ce9192ce9cb80ddef6', 'request_id': 'req-953c87fd-2a46-4e35-ba68-33d317f9ea9f'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:38.826071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-953c87fd-2a46-4e35-ba68-33d317f9ea9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea46ef8ffbdd41ce9192ce9cb80ddef6', 'request_id': 'req-953c87fd-2a46-4e35-ba68-33d317f9ea9f'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:38.831929 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-953c87fd-2a46-4e35-ba68-33d317f9ea9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea46ef8ffbdd41ce9192ce9cb80ddef6', 'request_id': 'req-953c87fd-2a46-4e35-ba68-33d317f9ea9f'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:38.845159 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-953c87fd-2a46-4e35-ba68-33d317f9ea9f admin admin] The token cache is being invalidate because project ea46ef8ffbdd41ce9192ce9cb80ddef6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:30:38.845562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-953c87fd-2a46-4e35-ba68-33d317f9ea9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:38.847092 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-953c87fd-2a46-4e35-ba68-33d317f9ea9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:38.848994 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1327/2637] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:30:38 2022] DELETE /identity/v3/projects/ea46ef8ffbdd41ce9192ce9cb80ddef6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:38.853775 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c2448-9faf-45bd-8ea7-3a91fe6c9d3b None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:38.853775 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c2448-9faf-45bd-8ea7-3a91fe6c9d3b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:38.854346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c2448-9faf-45bd-8ea7-3a91fe6c9d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:38.862740 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:38.879805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-dd8c2448-9faf-45bd-8ea7-3a91fe6c9d3b None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:38.881220 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-dd8c2448-9faf-45bd-8ea7-3a91fe6c9d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:38.943755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1311/2638] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:38.952133 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6521513-1a66-47aa-955d-a720f7031d03 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:38.952416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6521513-1a66-47aa-955d-a720f7031d03 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:38.952846 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6521513-1a66-47aa-955d-a720f7031d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:38.973142 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-e6521513-1a66-47aa-955d-a720f7031d03 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:38.974335 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e6521513-1a66-47aa-955d-a720f7031d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:38.983342 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1328/2639] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:38.987693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c2448-9faf-45bd-8ea7-3a91fe6c9d3b None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:38.998438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c2448-9faf-45bd-8ea7-3a91fe6c9d3b None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:39.003169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517ec5c-a905-4bd3-ac6d-a80e0678c188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:39.003783 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517ec5c-a905-4bd3-ac6d-a80e0678c188 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:39.004090 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517ec5c-a905-4bd3-ac6d-a80e0678c188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:39.004362 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517ec5c-a905-4bd3-ac6d-a80e0678c188 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:39.005009 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517ec5c-a905-4bd3-ac6d-a80e0678c188 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:30:39.005320 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517ec5c-a905-4bd3-ac6d-a80e0678c188 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:39.006952 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517ec5c-a905-4bd3-ac6d-a80e0678c188 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:39.018979 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1312/2640] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:30:39.023538 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6521513-1a66-47aa-955d-a720f7031d03 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:39.033537 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6521513-1a66-47aa-955d-a720f7031d03 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:39.038599 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f798d-546c-4527-b5f3-d6cc72b862b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:39.039224 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f798d-546c-4527-b5f3-d6cc72b862b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:39.039500 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f798d-546c-4527-b5f3-d6cc72b862b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:39.039836 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f798d-546c-4527-b5f3-d6cc72b862b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:39.040419 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f798d-546c-4527-b5f3-d6cc72b862b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:39.042284 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f798d-546c-4527-b5f3-d6cc72b862b4 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:39.069869 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1329/2641] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:30:39 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:30:39.074074 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517ec5c-a905-4bd3-ac6d-a80e0678c188 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:39.084469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517ec5c-a905-4bd3-ac6d-a80e0678c188 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:39.143550 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8a944-2c59-4ec9-9b72-361c70af252b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:39.143852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8a944-2c59-4ec9-9b72-361c70af252b admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:39.144308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8a944-2c59-4ec9-9b72-361c70af252b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:39.145488 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8a944-2c59-4ec9-9b72-361c70af252b admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:39.145488 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a8a944-2c59-4ec9-9b72-361c70af252b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:39.147282 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a8a944-2c59-4ec9-9b72-361c70af252b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:39.170445 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1313/2642] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:30:39 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:30:39.174597 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f798d-546c-4527-b5f3-d6cc72b862b4 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:39.186483 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f798d-546c-4527-b5f3-d6cc72b862b4 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:39.191570 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99659aaf-9d6b-4686-aadd-9dbf2ad1d94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:39.192373 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99659aaf-9d6b-4686-aadd-9dbf2ad1d94a admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:39.192723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99659aaf-9d6b-4686-aadd-9dbf2ad1d94a admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:39.193064 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99659aaf-9d6b-4686-aadd-9dbf2ad1d94a admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:39.193820 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99659aaf-9d6b-4686-aadd-9dbf2ad1d94a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:39.195472 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99659aaf-9d6b-4686-aadd-9dbf2ad1d94a admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:39.206393 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1330/2643] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:39.210457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8a944-2c59-4ec9-9b72-361c70af252b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:39.221267 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8a944-2c59-4ec9-9b72-361c70af252b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:39.226027 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd84962-d687-49d8-9b5a-983821083da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:39.226627 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd84962-d687-49d8-9b5a-983821083da3 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:39.226906 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd84962-d687-49d8-9b5a-983821083da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:39.227175 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd84962-d687-49d8-9b5a-983821083da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:39.227853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd84962-d687-49d8-9b5a-983821083da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:39.229484 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd84962-d687-49d8-9b5a-983821083da3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:39.241741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1314/2644] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:39.246361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99659aaf-9d6b-4686-aadd-9dbf2ad1d94a admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:39.256517 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99659aaf-9d6b-4686-aadd-9dbf2ad1d94a admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:39.261426 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75907e15-b9c3-46da-81c3-06fbabff2b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:39.262130 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75907e15-b9c3-46da-81c3-06fbabff2b6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:39.262431 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75907e15-b9c3-46da-81c3-06fbabff2b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:39.262701 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75907e15-b9c3-46da-81c3-06fbabff2b6c admin admin] PATH_INFO: `/v3/projects/a766e96bf6d04df59f5f1cabb43fe531/users/2a5daa6ef55c4bacb80b35f8ca5b8bc1/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:39.277741 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75907e15-b9c3-46da-81c3-06fbabff2b6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a766e96bf6d04df59f5f1cabb43fe531, user_id=2a5daa6ef55c4bacb80b35f8ca5b8bc1, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:39.281010 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75907e15-b9c3-46da-81c3-06fbabff2b6c admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:39.290577 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:39.299870 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1331/2645] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:39 2022] PUT /identity/v3/projects/a766e96bf6d04df59f5f1cabb43fe531/users/2a5daa6ef55c4bacb80b35f8ca5b8bc1/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:30:39.308180 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5115397-af55-4638-a3cf-281c10e5911c None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:39.308463 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5115397-af55-4638-a3cf-281c10e5911c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:39.308812 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5115397-af55-4638-a3cf-281c10e5911c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:39.342087 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a5115397-af55-4638-a3cf-281c10e5911c None None] MFA Rules not processed for user `2a5daa6ef55c4bacb80b35f8ca5b8bc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:39.343455 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a5115397-af55-4638-a3cf-281c10e5911c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:39.414496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:39.425584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1315/2646] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:39.426999 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75907e15-b9c3-46da-81c3-06fbabff2b6c admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:39.525564 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75907e15-b9c3-46da-81c3-06fbabff2b6c admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:39.533699 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf87d9-766f-4708-aaf9-1093aef2d4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:39.534313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf87d9-766f-4708-aaf9-1093aef2d4a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:39.534954 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf87d9-766f-4708-aaf9-1093aef2d4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:39.535223 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf87d9-766f-4708-aaf9-1093aef2d4a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:39.535903 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf87d9-766f-4708-aaf9-1093aef2d4a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:39.538125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf87d9-766f-4708-aaf9-1093aef2d4a0 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:39.574732 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1332/2647] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:30:39 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:30:39.580182 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5115397-af55-4638-a3cf-281c10e5911c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:39.595402 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5115397-af55-4638-a3cf-281c10e5911c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:39.677052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc4afe-4110-440e-8238-afbe1d1bed10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:39.677746 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc4afe-4110-440e-8238-afbe1d1bed10 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:39.678066 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc4afe-4110-440e-8238-afbe1d1bed10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:39.678301 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc4afe-4110-440e-8238-afbe1d1bed10 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:39.678963 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc4afe-4110-440e-8238-afbe1d1bed10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:39.681084 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc4afe-4110-440e-8238-afbe1d1bed10 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:39.707538 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1316/2648] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:30:39 2022] POST /identity/v3/users => generated 297 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:30:39.711752 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf87d9-766f-4708-aaf9-1093aef2d4a0 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:39.723464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf87d9-766f-4708-aaf9-1093aef2d4a0 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:39.728133 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd8c7b-a193-4c87-b096-3a572b754d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:39.728732 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd8c7b-a193-4c87-b096-3a572b754d1f admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:39.729006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd8c7b-a193-4c87-b096-3a572b754d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:39.729266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd8c7b-a193-4c87-b096-3a572b754d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:39.729991 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd8c7b-a193-4c87-b096-3a572b754d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:39.731820 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd8c7b-a193-4c87-b096-3a572b754d1f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:39.743756 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1333/2649] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:39.748370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc4afe-4110-440e-8238-afbe1d1bed10 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:39.758194 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc4afe-4110-440e-8238-afbe1d1bed10 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:39.763834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d247e2-3381-4021-9baa-4d5e8675b3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:39.764511 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d247e2-3381-4021-9baa-4d5e8675b3f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:39.764788 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d247e2-3381-4021-9baa-4d5e8675b3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:39.765052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d247e2-3381-4021-9baa-4d5e8675b3f8 admin admin] PATH_INFO: `/v3/projects/38de56e1e3194fa8b70750d88ea50a70/users/de0bd304dd5044ca9620af935776cb86/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:39.781961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d247e2-3381-4021-9baa-4d5e8675b3f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38de56e1e3194fa8b70750d88ea50a70, user_id=de0bd304dd5044ca9620af935776cb86, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:39.784678 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d247e2-3381-4021-9baa-4d5e8675b3f8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:39.798424 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1317/2650] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:39 2022] PUT /identity/v3/projects/38de56e1e3194fa8b70750d88ea50a70/users/de0bd304dd5044ca9620af935776cb86/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:39.802556 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd8c7b-a193-4c87-b096-3a572b754d1f admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:39.814104 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd8c7b-a193-4c87-b096-3a572b754d1f admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:39.878079 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dad116-6aca-4681-929f-b2e8fc1095ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:39.878716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dad116-6aca-4681-929f-b2e8fc1095ee admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:39.878974 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dad116-6aca-4681-929f-b2e8fc1095ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:39.879246 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dad116-6aca-4681-929f-b2e8fc1095ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:39.879916 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dad116-6aca-4681-929f-b2e8fc1095ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:39.881527 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dad116-6aca-4681-929f-b2e8fc1095ee admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:39.892175 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1334/2651] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:39.896799 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d247e2-3381-4021-9baa-4d5e8675b3f8 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:39.913052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d247e2-3381-4021-9baa-4d5e8675b3f8 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:39.918658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e3fe3-22c1-4589-9bb0-1def235210d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:39.919952 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e3fe3-22c1-4589-9bb0-1def235210d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:39.920291 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e3fe3-22c1-4589-9bb0-1def235210d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:39.920804 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e3fe3-22c1-4589-9bb0-1def235210d2 admin admin] PATH_INFO: `/v3/projects/38de56e1e3194fa8b70750d88ea50a70/users/de0bd304dd5044ca9620af935776cb86/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:39.922662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64e3fe3-22c1-4589-9bb0-1def235210d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38de56e1e3194fa8b70750d88ea50a70, user_id=de0bd304dd5044ca9620af935776cb86, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:39.925650 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64e3fe3-22c1-4589-9bb0-1def235210d2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:39.938542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1318/2652] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:39 2022] PUT /identity/v3/projects/38de56e1e3194fa8b70750d88ea50a70/users/de0bd304dd5044ca9620af935776cb86/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:30:39.956264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a0ba2-531f-42f6-978e-bbfd34a72850 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:39.956659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a0ba2-531f-42f6-978e-bbfd34a72850 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:39.956817 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a0ba2-531f-42f6-978e-bbfd34a72850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:39.992409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-2e1a0ba2-531f-42f6-978e-bbfd34a72850 None None] MFA Rules not processed for user `de0bd304dd5044ca9620af935776cb86`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:39.993591 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2e1a0ba2-531f-42f6-978e-bbfd34a72850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:40.060930 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:40.075943 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1335/2653] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:39 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:40.086898 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e3fe3-22c1-4589-9bb0-1def235210d2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:40.089464 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a64e3fe3-22c1-4589-9bb0-1def235210d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:40.165672 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e3fe3-22c1-4589-9bb0-1def235210d2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:40.170972 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a34f7-34f1-42c2-9d78-51ed861537a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:40.171570 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a34f7-34f1-42c2-9d78-51ed861537a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:40.171896 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a34f7-34f1-42c2-9d78-51ed861537a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:40.172163 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a34f7-34f1-42c2-9d78-51ed861537a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:40.182115 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a34f7-34f1-42c2-9d78-51ed861537a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:40.183617 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a34f7-34f1-42c2-9d78-51ed861537a3 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:40.193525 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1319/2654] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:40.512268 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a0ba2-531f-42f6-978e-bbfd34a72850 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:40.521589 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a0ba2-531f-42f6-978e-bbfd34a72850 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:40.526449 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43e38c-3ffb-4d6f-9e11-7ba81e6ceb3e tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de0bd304dd5044ca9620af935776cb86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38de56e1e3194fa8b70750d88ea50a70', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:40.527050 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43e38c-3ffb-4d6f-9e11-7ba81e6ceb3e tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:40.527332 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43e38c-3ffb-4d6f-9e11-7ba81e6ceb3e tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:40.527608 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43e38c-3ffb-4d6f-9e11-7ba81e6ceb3e tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:40.528315 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e43e38c-3ffb-4d6f-9e11-7ba81e6ceb3e tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:30:40.529190 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e43e38c-3ffb-4d6f-9e11-7ba81e6ceb3e tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:40.530807 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e43e38c-3ffb-4d6f-9e11-7ba81e6ceb3e tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:40.542207 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1336/2655] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:30:40.546943 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a34f7-34f1-42c2-9d78-51ed861537a3 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:40.556143 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a34f7-34f1-42c2-9d78-51ed861537a3 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:40.560715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac87a33-08b6-46b3-834f-2c3900bbe4b7 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de0bd304dd5044ca9620af935776cb86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38de56e1e3194fa8b70750d88ea50a70', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:40.561299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac87a33-08b6-46b3-834f-2c3900bbe4b7 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:40.561578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac87a33-08b6-46b3-834f-2c3900bbe4b7 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:40.561850 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac87a33-08b6-46b3-834f-2c3900bbe4b7 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:40.562473 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac87a33-08b6-46b3-834f-2c3900bbe4b7 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:40.564398 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac87a33-08b6-46b3-834f-2c3900bbe4b7 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:40.592586 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1320/2656] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:30:40 2022] POST /identity/v3/projects => generated 355 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:30:40.596667 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43e38c-3ffb-4d6f-9e11-7ba81e6ceb3e tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:40.609312 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43e38c-3ffb-4d6f-9e11-7ba81e6ceb3e tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:40.670266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c26708-bca0-49ff-8630-fd2bfaead1f3 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de0bd304dd5044ca9620af935776cb86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38de56e1e3194fa8b70750d88ea50a70', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:40.671062 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c26708-bca0-49ff-8630-fd2bfaead1f3 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:40.671347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c26708-bca0-49ff-8630-fd2bfaead1f3 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:40.671620 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c26708-bca0-49ff-8630-fd2bfaead1f3 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:40.672298 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c26708-bca0-49ff-8630-fd2bfaead1f3 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:30:40.672719 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c26708-bca0-49ff-8630-fd2bfaead1f3 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:40.674421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c26708-bca0-49ff-8630-fd2bfaead1f3 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:40.688378 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1337/2657] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:30:40.739402 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac87a33-08b6-46b3-834f-2c3900bbe4b7 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:40.751927 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac87a33-08b6-46b3-834f-2c3900bbe4b7 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:40.756826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87153162-27fe-4e14-856c-53d5664949e1 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de0bd304dd5044ca9620af935776cb86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38de56e1e3194fa8b70750d88ea50a70', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:40.757530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87153162-27fe-4e14-856c-53d5664949e1 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:40.757884 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87153162-27fe-4e14-856c-53d5664949e1 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:40.758243 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87153162-27fe-4e14-856c-53d5664949e1 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] PATH_INFO: `/v3/projects/31d697feed3a46f7bbda876e7def4715` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:40.759478 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87153162-27fe-4e14-856c-53d5664949e1 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] RBAC: Authorizing `identity:delete_project(project_id=31d697feed3a46f7bbda876e7def4715)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:40.761592 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87153162-27fe-4e14-856c-53d5664949e1 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:40.796208 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:40.818872 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-87153162-27fe-4e14-856c-53d5664949e1 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31d697feed3a46f7bbda876e7def4715', 'request_id': 'req-87153162-27fe-4e14-856c-53d5664949e1'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:40.826931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-87153162-27fe-4e14-856c-53d5664949e1 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31d697feed3a46f7bbda876e7def4715', 'request_id': 'req-87153162-27fe-4e14-856c-53d5664949e1'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:40.833334 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-87153162-27fe-4e14-856c-53d5664949e1 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31d697feed3a46f7bbda876e7def4715', 'request_id': 'req-87153162-27fe-4e14-856c-53d5664949e1'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:40.846942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-87153162-27fe-4e14-856c-53d5664949e1 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] The token cache is being invalidate because project 31d697feed3a46f7bbda876e7def4715 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:30:40.847232 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-87153162-27fe-4e14-856c-53d5664949e1 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:40.847936 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-87153162-27fe-4e14-856c-53d5664949e1 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:40.849415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1321/2658] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:30:40 2022] DELETE /identity/v3/projects/31d697feed3a46f7bbda876e7def4715 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:41.040642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c26708-bca0-49ff-8630-fd2bfaead1f3 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:41.042365 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c2c26708-bca0-49ff-8630-fd2bfaead1f3 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:41.107139 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c26708-bca0-49ff-8630-fd2bfaead1f3 tempest-VolumeQuotaClassesTest-1645196506 tempest-VolumeQuotaClassesTest-1645196506-project-admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:41.161337 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac07cfe-7df8-4bda-bd02-9c31de71760a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:41.161946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac07cfe-7df8-4bda-bd02-9c31de71760a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:41.162222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac07cfe-7df8-4bda-bd02-9c31de71760a admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:41.162485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac07cfe-7df8-4bda-bd02-9c31de71760a admin admin] PATH_INFO: `/v3/users/2a5daa6ef55c4bacb80b35f8ca5b8bc1` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:41.163631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac07cfe-7df8-4bda-bd02-9c31de71760a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a5daa6ef55c4bacb80b35f8ca5b8bc1)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:41.165545 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac07cfe-7df8-4bda-bd02-9c31de71760a admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:41.193188 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:41.218127 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-aac07cfe-7df8-4bda-bd02-9c31de71760a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a5daa6ef55c4bacb80b35f8ca5b8bc1'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:41.218831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-aac07cfe-7df8-4bda-bd02-9c31de71760a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a5daa6ef55c4bacb80b35f8ca5b8bc1'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:41.219507 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-aac07cfe-7df8-4bda-bd02-9c31de71760a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a5daa6ef55c4bacb80b35f8ca5b8bc1'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:41.226507 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-aac07cfe-7df8-4bda-bd02-9c31de71760a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a5daa6ef55c4bacb80b35f8ca5b8bc1'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:41.239168 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-aac07cfe-7df8-4bda-bd02-9c31de71760a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a5daa6ef55c4bacb80b35f8ca5b8bc1'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:41.252378 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1338/2659] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:30:41 2022] DELETE /identity/v3/users/2a5daa6ef55c4bacb80b35f8ca5b8bc1 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:41.256914 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c176fcf-86b7-4b7e-b772-3335f098ae2f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:41.257420 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c176fcf-86b7-4b7e-b772-3335f098ae2f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:41.257757 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c176fcf-86b7-4b7e-b772-3335f098ae2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:41.276170 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-5c176fcf-86b7-4b7e-b772-3335f098ae2f None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:41.277382 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5c176fcf-86b7-4b7e-b772-3335f098ae2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:41.344069 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1322/2660] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:41.354045 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac07cfe-7df8-4bda-bd02-9c31de71760a admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:41.355574 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-aac07cfe-7df8-4bda-bd02-9c31de71760a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:41.422040 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac07cfe-7df8-4bda-bd02-9c31de71760a admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:41.425764 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391448b9-aa5b-406b-acb8-b7a6c05161a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:41.426435 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391448b9-aa5b-406b-acb8-b7a6c05161a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:41.426783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391448b9-aa5b-406b-acb8-b7a6c05161a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:41.427122 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391448b9-aa5b-406b-acb8-b7a6c05161a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:41.436413 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391448b9-aa5b-406b-acb8-b7a6c05161a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:41.437844 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391448b9-aa5b-406b-acb8-b7a6c05161a8 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:41.448127 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1339/2661] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:30:41.750248 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c176fcf-86b7-4b7e-b772-3335f098ae2f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:41.751691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-5c176fcf-86b7-4b7e-b772-3335f098ae2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:41.773763 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c176fcf-86b7-4b7e-b772-3335f098ae2f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:41.778282 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cbeda-2283-4820-a28d-9fa4d28b62b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:41.778912 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cbeda-2283-4820-a28d-9fa4d28b62b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:41.779189 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cbeda-2283-4820-a28d-9fa4d28b62b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:41.779459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cbeda-2283-4820-a28d-9fa4d28b62b5 admin admin] PATH_INFO: `/v3/projects/a766e96bf6d04df59f5f1cabb43fe531` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:41.780612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5cbeda-2283-4820-a28d-9fa4d28b62b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a766e96bf6d04df59f5f1cabb43fe531)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:41.782544 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5cbeda-2283-4820-a28d-9fa4d28b62b5 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:41.829099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-de5cbeda-2283-4820-a28d-9fa4d28b62b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a766e96bf6d04df59f5f1cabb43fe531', 'request_id': 'req-de5cbeda-2283-4820-a28d-9fa4d28b62b5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:41.835607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-de5cbeda-2283-4820-a28d-9fa4d28b62b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a766e96bf6d04df59f5f1cabb43fe531', 'request_id': 'req-de5cbeda-2283-4820-a28d-9fa4d28b62b5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:41.841362 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-de5cbeda-2283-4820-a28d-9fa4d28b62b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a766e96bf6d04df59f5f1cabb43fe531', 'request_id': 'req-de5cbeda-2283-4820-a28d-9fa4d28b62b5'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:41.853892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-de5cbeda-2283-4820-a28d-9fa4d28b62b5 admin admin] The token cache is being invalidate because project a766e96bf6d04df59f5f1cabb43fe531 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:30:41.854188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-de5cbeda-2283-4820-a28d-9fa4d28b62b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:41.854817 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-de5cbeda-2283-4820-a28d-9fa4d28b62b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:41.857012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1323/2662] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:30:41 2022] DELETE /identity/v3/projects/a766e96bf6d04df59f5f1cabb43fe531 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:41.861055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391448b9-aa5b-406b-acb8-b7a6c05161a8 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:41.870612 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391448b9-aa5b-406b-acb8-b7a6c05161a8 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:41.927933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58687e62-ee27-4413-863d-b1e479b63f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:41.928523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58687e62-ee27-4413-863d-b1e479b63f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:41.928831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58687e62-ee27-4413-863d-b1e479b63f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:41.929097 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58687e62-ee27-4413-863d-b1e479b63f10 admin admin] PATH_INFO: `/v3/users/de0bd304dd5044ca9620af935776cb86` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:41.930263 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58687e62-ee27-4413-863d-b1e479b63f10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de0bd304dd5044ca9620af935776cb86)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:41.932214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58687e62-ee27-4413-863d-b1e479b63f10 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:41.979222 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:41.981242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-58687e62-ee27-4413-863d-b1e479b63f10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de0bd304dd5044ca9620af935776cb86'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:41.982557 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-58687e62-ee27-4413-863d-b1e479b63f10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de0bd304dd5044ca9620af935776cb86'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:41.983628 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-58687e62-ee27-4413-863d-b1e479b63f10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de0bd304dd5044ca9620af935776cb86'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:41.992981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-58687e62-ee27-4413-863d-b1e479b63f10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de0bd304dd5044ca9620af935776cb86'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:42.005384 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-58687e62-ee27-4413-863d-b1e479b63f10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de0bd304dd5044ca9620af935776cb86'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:42.018627 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1340/2663] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:30:41 2022] DELETE /identity/v3/users/de0bd304dd5044ca9620af935776cb86 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:42.290751 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cbeda-2283-4820-a28d-9fa4d28b62b5 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:42.293965 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-de5cbeda-2283-4820-a28d-9fa4d28b62b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:42.318548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cbeda-2283-4820-a28d-9fa4d28b62b5 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:42.366041 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:42.396160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac325635-d79a-49bd-a81c-69d9ecbc0c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:42.396975 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac325635-d79a-49bd-a81c-69d9ecbc0c48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:42.397263 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac325635-d79a-49bd-a81c-69d9ecbc0c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:42.397535 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac325635-d79a-49bd-a81c-69d9ecbc0c48 admin admin] PATH_INFO: `/v3/projects/38de56e1e3194fa8b70750d88ea50a70` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:42.398685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac325635-d79a-49bd-a81c-69d9ecbc0c48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38de56e1e3194fa8b70750d88ea50a70)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:42.400611 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac325635-d79a-49bd-a81c-69d9ecbc0c48 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:42.472215 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ac325635-d79a-49bd-a81c-69d9ecbc0c48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38de56e1e3194fa8b70750d88ea50a70', 'request_id': 'req-ac325635-d79a-49bd-a81c-69d9ecbc0c48'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:42.481917 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ac325635-d79a-49bd-a81c-69d9ecbc0c48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38de56e1e3194fa8b70750d88ea50a70', 'request_id': 'req-ac325635-d79a-49bd-a81c-69d9ecbc0c48'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:42.488916 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ac325635-d79a-49bd-a81c-69d9ecbc0c48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38de56e1e3194fa8b70750d88ea50a70', 'request_id': 'req-ac325635-d79a-49bd-a81c-69d9ecbc0c48'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:42.504948 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ac325635-d79a-49bd-a81c-69d9ecbc0c48 admin admin] The token cache is being invalidate because project 38de56e1e3194fa8b70750d88ea50a70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:30:42.505369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ac325635-d79a-49bd-a81c-69d9ecbc0c48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:42.506205 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ac325635-d79a-49bd-a81c-69d9ecbc0c48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:42.507849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1324/2664] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:30:42 2022] DELETE /identity/v3/projects/38de56e1e3194fa8b70750d88ea50a70 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:42.514526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a38ade-9095-40fd-ab24-f5d7c8bcf6e9 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:42.515036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a38ade-9095-40fd-ab24-f5d7c8bcf6e9 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:42.515124 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a38ade-9095-40fd-ab24-f5d7c8bcf6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:42.542839 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-d3a38ade-9095-40fd-ab24-f5d7c8bcf6e9 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:42.544768 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-d3a38ade-9095-40fd-ab24-f5d7c8bcf6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:42.623443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1341/2665] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:42.624931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee006a-c26e-48fc-b758-bf57fccfe0d5 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:42.625213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee006a-c26e-48fc-b758-bf57fccfe0d5 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:42.625487 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee006a-c26e-48fc-b758-bf57fccfe0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:42.648893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-33ee006a-c26e-48fc-b758-bf57fccfe0d5 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:42.650091 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-33ee006a-c26e-48fc-b758-bf57fccfe0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:42.662808 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1325/2666] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:42.672653 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a38ade-9095-40fd-ab24-f5d7c8bcf6e9 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:42.679023 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a38ade-9095-40fd-ab24-f5d7c8bcf6e9 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:42.684211 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f5ad5-0cd7-4141-ad2f-9777af4b0a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:42.685503 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f5ad5-0cd7-4141-ad2f-9777af4b0a6f admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:42.686555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f5ad5-0cd7-4141-ad2f-9777af4b0a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:42.686832 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f5ad5-0cd7-4141-ad2f-9777af4b0a6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:42.687480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f5ad5-0cd7-4141-ad2f-9777af4b0a6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:30:42.687796 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f5ad5-0cd7-4141-ad2f-9777af4b0a6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:42.689389 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f5ad5-0cd7-4141-ad2f-9777af4b0a6f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:42.704326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1342/2667] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:30:42.708178 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee006a-c26e-48fc-b758-bf57fccfe0d5 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:42.718107 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee006a-c26e-48fc-b758-bf57fccfe0d5 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:42.722742 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30000f21-daf1-43ec-80ad-a227b32b8377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:42.723284 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30000f21-daf1-43ec-80ad-a227b32b8377 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:42.723563 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30000f21-daf1-43ec-80ad-a227b32b8377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:42.723874 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30000f21-daf1-43ec-80ad-a227b32b8377 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:42.724673 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30000f21-daf1-43ec-80ad-a227b32b8377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:42.726578 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30000f21-daf1-43ec-80ad-a227b32b8377 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:42.759694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1326/2668] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:30:42 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:30:42.764614 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f5ad5-0cd7-4141-ad2f-9777af4b0a6f admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:42.774741 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f5ad5-0cd7-4141-ad2f-9777af4b0a6f admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:42.835643 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d0a66-0147-44b2-a807-cf525135c570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:42.836288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d0a66-0147-44b2-a807-cf525135c570 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:42.836569 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d0a66-0147-44b2-a807-cf525135c570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:42.836851 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d0a66-0147-44b2-a807-cf525135c570 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:42.837470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d0a66-0147-44b2-a807-cf525135c570 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:42.839371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d0a66-0147-44b2-a807-cf525135c570 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:42.859903 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1343/2669] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:30:42 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:30:42.864177 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30000f21-daf1-43ec-80ad-a227b32b8377 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:42.874946 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30000f21-daf1-43ec-80ad-a227b32b8377 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:42.879515 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecad7ef-5da5-4ee0-9c2e-d849e0d074b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:42.880137 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecad7ef-5da5-4ee0-9c2e-d849e0d074b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:42.880415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecad7ef-5da5-4ee0-9c2e-d849e0d074b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:42.880710 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecad7ef-5da5-4ee0-9c2e-d849e0d074b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:42.881353 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecad7ef-5da5-4ee0-9c2e-d849e0d074b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:42.882935 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecad7ef-5da5-4ee0-9c2e-d849e0d074b7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:42.893984 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1327/2670] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:42.898196 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d0a66-0147-44b2-a807-cf525135c570 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:42.908308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d0a66-0147-44b2-a807-cf525135c570 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:42.913002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86802b9d-7f25-483b-a60a-cb3d228421c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:42.913588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86802b9d-7f25-483b-a60a-cb3d228421c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:42.913869 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86802b9d-7f25-483b-a60a-cb3d228421c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:42.914143 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86802b9d-7f25-483b-a60a-cb3d228421c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:42.914779 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86802b9d-7f25-483b-a60a-cb3d228421c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:42.916399 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86802b9d-7f25-483b-a60a-cb3d228421c0 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:42.928470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1344/2671] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:42.932584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecad7ef-5da5-4ee0-9c2e-d849e0d074b7 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:42.942805 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecad7ef-5da5-4ee0-9c2e-d849e0d074b7 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:42.947686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3370f-4143-4b8c-a385-afb34baffd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:42.948347 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3370f-4143-4b8c-a385-afb34baffd45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:42.948624 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3370f-4143-4b8c-a385-afb34baffd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:42.948893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3370f-4143-4b8c-a385-afb34baffd45 admin admin] PATH_INFO: `/v3/projects/cf1f6681ca1042a4ad10e05c2474d7f0/users/d1c948c1f3b9449bbe06db1cd4a9ef59/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:42.964754 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3370f-4143-4b8c-a385-afb34baffd45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf1f6681ca1042a4ad10e05c2474d7f0, user_id=d1c948c1f3b9449bbe06db1cd4a9ef59, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:42.967356 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3370f-4143-4b8c-a385-afb34baffd45 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:42.979910 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1328/2672] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:42 2022] PUT /identity/v3/projects/cf1f6681ca1042a4ad10e05c2474d7f0/users/d1c948c1f3b9449bbe06db1cd4a9ef59/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:42.990583 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883293a4-f5a5-416d-8b42-ef7a99bf28d1 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:42.990867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883293a4-f5a5-416d-8b42-ef7a99bf28d1 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:42.991136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883293a4-f5a5-416d-8b42-ef7a99bf28d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:43.022724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-883293a4-f5a5-416d-8b42-ef7a99bf28d1 None None] MFA Rules not processed for user `d1c948c1f3b9449bbe06db1cd4a9ef59`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:43.022724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-883293a4-f5a5-416d-8b42-ef7a99bf28d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:43.040640 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:43.092020 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1345/2673] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:43.096608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3370f-4143-4b8c-a385-afb34baffd45 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:43.158675 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3370f-4143-4b8c-a385-afb34baffd45 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:43.163482 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032d210-33ea-4efe-88f2-def217ae4d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:43.164121 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032d210-33ea-4efe-88f2-def217ae4d4b admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:43.164405 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032d210-33ea-4efe-88f2-def217ae4d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:43.164679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032d210-33ea-4efe-88f2-def217ae4d4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:43.165357 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032d210-33ea-4efe-88f2-def217ae4d4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:43.167248 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032d210-33ea-4efe-88f2-def217ae4d4b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:43.177903 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:43.196671 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1329/2674] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:30:43 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:30:43.201502 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883293a4-f5a5-416d-8b42-ef7a99bf28d1 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:43.213780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883293a4-f5a5-416d-8b42-ef7a99bf28d1 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:43.268717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70133a1-989e-4f72-ae2f-de3f38bf8b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:43.269232 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70133a1-989e-4f72-ae2f-de3f38bf8b12 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:43.269510 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70133a1-989e-4f72-ae2f-de3f38bf8b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:43.269781 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70133a1-989e-4f72-ae2f-de3f38bf8b12 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:43.270404 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70133a1-989e-4f72-ae2f-de3f38bf8b12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:43.272370 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70133a1-989e-4f72-ae2f-de3f38bf8b12 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:43.293246 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1346/2675] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:30:43 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:30:43.297191 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032d210-33ea-4efe-88f2-def217ae4d4b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:43.307684 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032d210-33ea-4efe-88f2-def217ae4d4b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:43.312597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdfd2c4-2ec9-4910-b432-5da171a178fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:43.313193 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdfd2c4-2ec9-4910-b432-5da171a178fc admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:43.313470 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdfd2c4-2ec9-4910-b432-5da171a178fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:43.313739 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdfd2c4-2ec9-4910-b432-5da171a178fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:43.314370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdfd2c4-2ec9-4910-b432-5da171a178fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:43.316036 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdfd2c4-2ec9-4910-b432-5da171a178fc admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:43.326586 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1330/2676] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:43.330828 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70133a1-989e-4f72-ae2f-de3f38bf8b12 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:43.341591 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70133a1-989e-4f72-ae2f-de3f38bf8b12 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:43.346180 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4e7be-dd17-4610-baf5-985b78098565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:43.346823 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4e7be-dd17-4610-baf5-985b78098565 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:43.347102 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4e7be-dd17-4610-baf5-985b78098565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:43.347400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4e7be-dd17-4610-baf5-985b78098565 admin admin] PATH_INFO: `/v3/projects/20944d9e8b5d41f8a8c8cb402660820a/users/af65cb1aeef849d090ecd01f4507392c/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:43.361761 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4e7be-dd17-4610-baf5-985b78098565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20944d9e8b5d41f8a8c8cb402660820a, user_id=af65cb1aeef849d090ecd01f4507392c, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:43.364451 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4e7be-dd17-4610-baf5-985b78098565 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:43.377006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1347/2677] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:43 2022] PUT /identity/v3/projects/20944d9e8b5d41f8a8c8cb402660820a/users/af65cb1aeef849d090ecd01f4507392c/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:30:43.380960 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdfd2c4-2ec9-4910-b432-5da171a178fc admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:43.391413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdfd2c4-2ec9-4910-b432-5da171a178fc admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:43.448098 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eae7fa-0242-4d94-a11d-dfe424a1b679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:43.448689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eae7fa-0242-4d94-a11d-dfe424a1b679 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:43.448965 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eae7fa-0242-4d94-a11d-dfe424a1b679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:43.449237 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eae7fa-0242-4d94-a11d-dfe424a1b679 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:43.449867 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eae7fa-0242-4d94-a11d-dfe424a1b679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:43.451747 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eae7fa-0242-4d94-a11d-dfe424a1b679 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:43.461277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1331/2678] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:43.465483 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4e7be-dd17-4610-baf5-985b78098565 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:43.476078 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4e7be-dd17-4610-baf5-985b78098565 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:43.480928 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40148271-ed48-493f-b771-d7fd3845c233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:43.482080 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40148271-ed48-493f-b771-d7fd3845c233 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:43.482392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40148271-ed48-493f-b771-d7fd3845c233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:43.482741 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40148271-ed48-493f-b771-d7fd3845c233 admin admin] PATH_INFO: `/v3/projects/20944d9e8b5d41f8a8c8cb402660820a/users/af65cb1aeef849d090ecd01f4507392c/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:43.484578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40148271-ed48-493f-b771-d7fd3845c233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20944d9e8b5d41f8a8c8cb402660820a, user_id=af65cb1aeef849d090ecd01f4507392c, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:43.487234 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40148271-ed48-493f-b771-d7fd3845c233 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:43.498016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1348/2679] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:43 2022] PUT /identity/v3/projects/20944d9e8b5d41f8a8c8cb402660820a/users/af65cb1aeef849d090ecd01f4507392c/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:43.508926 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af77e17-1ed1-49ae-855b-581a5b756c0c None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:43.509214 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af77e17-1ed1-49ae-855b-581a5b756c0c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:43.509489 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af77e17-1ed1-49ae-855b-581a5b756c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:43.537156 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-3af77e17-1ed1-49ae-855b-581a5b756c0c None None] MFA Rules not processed for user `af65cb1aeef849d090ecd01f4507392c`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:43.538273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3af77e17-1ed1-49ae-855b-581a5b756c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:43.605397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1332/2680] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:43.616598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40148271-ed48-493f-b771-d7fd3845c233 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:43.618387 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-40148271-ed48-493f-b771-d7fd3845c233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:43.676656 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40148271-ed48-493f-b771-d7fd3845c233 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:43.680306 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b617c2-9513-4eae-8e69-6904d96011c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:43.680899 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b617c2-9513-4eae-8e69-6904d96011c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:43.681173 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b617c2-9513-4eae-8e69-6904d96011c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:43.681440 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b617c2-9513-4eae-8e69-6904d96011c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:43.705340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:43.737647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b617c2-9513-4eae-8e69-6904d96011c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:43.739064 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b617c2-9513-4eae-8e69-6904d96011c9 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:43.748308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1349/2681] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:30:44.903254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af77e17-1ed1-49ae-855b-581a5b756c0c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:44.904711 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3af77e17-1ed1-49ae-855b-581a5b756c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:44.963074 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af77e17-1ed1-49ae-855b-581a5b756c0c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:44.966780 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a55a3-9218-4285-b83a-a7076206d1fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:44.967393 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a55a3-9218-4285-b83a-a7076206d1fb service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:44.967673 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a55a3-9218-4285-b83a-a7076206d1fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:44.968042 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a55a3-9218-4285-b83a-a7076206d1fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:44.970106 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ea9a55a3-9218-4285-b83a-a7076206d1fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:44.985100 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:45.049524 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a55a3-9218-4285-b83a-a7076206d1fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:45.050975 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a55a3-9218-4285-b83a-a7076206d1fb service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:45.060762 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1333/2682] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:46.388141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b617c2-9513-4eae-8e69-6904d96011c9 service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:46.494452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b617c2-9513-4eae-8e69-6904d96011c9 service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:46.507391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabe7d5-8d5a-4253-ab95-e22f611ff732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:46.508760 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabe7d5-8d5a-4253-ab95-e22f611ff732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:46.509472 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabe7d5-8d5a-4253-ab95-e22f611ff732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:46.509938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabe7d5-8d5a-4253-ab95-e22f611ff732 admin admin] PATH_INFO: `/v3/users/d1c948c1f3b9449bbe06db1cd4a9ef59` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:46.511475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabe7d5-8d5a-4253-ab95-e22f611ff732 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c948c1f3b9449bbe06db1cd4a9ef59)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:46.514366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabe7d5-8d5a-4253-ab95-e22f611ff732 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:46.576360 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ffabe7d5-8d5a-4253-ab95-e22f611ff732 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c948c1f3b9449bbe06db1cd4a9ef59'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:46.577169 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ffabe7d5-8d5a-4253-ab95-e22f611ff732 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c948c1f3b9449bbe06db1cd4a9ef59'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:46.577916 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ffabe7d5-8d5a-4253-ab95-e22f611ff732 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c948c1f3b9449bbe06db1cd4a9ef59'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:46.586365 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ffabe7d5-8d5a-4253-ab95-e22f611ff732 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c948c1f3b9449bbe06db1cd4a9ef59'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:46.606907 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ffabe7d5-8d5a-4253-ab95-e22f611ff732 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c948c1f3b9449bbe06db1cd4a9ef59'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:46.628170 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1350/2683] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:30:46 2022] DELETE /identity/v3/users/d1c948c1f3b9449bbe06db1cd4a9ef59 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:46.632853 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70285766-84f1-4e47-b55e-84c1142a55c9 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:46.633733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70285766-84f1-4e47-b55e-84c1142a55c9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:46.634551 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70285766-84f1-4e47-b55e-84c1142a55c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:46.687533 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-70285766-84f1-4e47-b55e-84c1142a55c9 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:46.689301 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-70285766-84f1-4e47-b55e-84c1142a55c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:46.771185 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1334/2684] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:46.781060 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabe7d5-8d5a-4253-ab95-e22f611ff732 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:46.782468 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ffabe7d5-8d5a-4253-ab95-e22f611ff732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:46.833576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:46.847058 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabe7d5-8d5a-4253-ab95-e22f611ff732 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:46.852713 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335a8a5-613d-4cd2-891c-b0a98c2d5970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:46.853742 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335a8a5-613d-4cd2-891c-b0a98c2d5970 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:46.854181 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335a8a5-613d-4cd2-891c-b0a98c2d5970 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:46.854713 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335a8a5-613d-4cd2-891c-b0a98c2d5970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:46.865440 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335a8a5-613d-4cd2-891c-b0a98c2d5970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:46.867187 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335a8a5-613d-4cd2-891c-b0a98c2d5970 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:46.879071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1351/2685] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:47.159235 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70285766-84f1-4e47-b55e-84c1142a55c9 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:47.160858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-70285766-84f1-4e47-b55e-84c1142a55c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:47.174895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70285766-84f1-4e47-b55e-84c1142a55c9 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:47.180708 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b8a5a-4b2c-4adb-a5df-c887c364f968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:47.181438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b8a5a-4b2c-4adb-a5df-c887c364f968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:47.181707 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b8a5a-4b2c-4adb-a5df-c887c364f968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:47.181985 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b8a5a-4b2c-4adb-a5df-c887c364f968 admin admin] PATH_INFO: `/v3/projects/cf1f6681ca1042a4ad10e05c2474d7f0` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:47.183141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b8a5a-4b2c-4adb-a5df-c887c364f968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf1f6681ca1042a4ad10e05c2474d7f0)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:47.185780 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b8a5a-4b2c-4adb-a5df-c887c364f968 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:47.240393 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-050b8a5a-4b2c-4adb-a5df-c887c364f968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf1f6681ca1042a4ad10e05c2474d7f0', 'request_id': 'req-050b8a5a-4b2c-4adb-a5df-c887c364f968'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:47.248569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-050b8a5a-4b2c-4adb-a5df-c887c364f968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf1f6681ca1042a4ad10e05c2474d7f0', 'request_id': 'req-050b8a5a-4b2c-4adb-a5df-c887c364f968'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:47.258644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-050b8a5a-4b2c-4adb-a5df-c887c364f968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf1f6681ca1042a4ad10e05c2474d7f0', 'request_id': 'req-050b8a5a-4b2c-4adb-a5df-c887c364f968'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:47.273042 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-050b8a5a-4b2c-4adb-a5df-c887c364f968 admin admin] The token cache is being invalidate because project cf1f6681ca1042a4ad10e05c2474d7f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:30:47.273412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-050b8a5a-4b2c-4adb-a5df-c887c364f968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:47.274119 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-050b8a5a-4b2c-4adb-a5df-c887c364f968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:47.275737 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1335/2686] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:30:47 2022] DELETE /identity/v3/projects/cf1f6681ca1042a4ad10e05c2474d7f0 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:47.280393 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335a8a5-613d-4cd2-891c-b0a98c2d5970 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:47.290045 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335a8a5-613d-4cd2-891c-b0a98c2d5970 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:47.351726 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c37565-5e21-4db8-ad79-9279700d0262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:47.352512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37565-5e21-4db8-ad79-9279700d0262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:47.352798 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37565-5e21-4db8-ad79-9279700d0262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:47.353071 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37565-5e21-4db8-ad79-9279700d0262 admin admin] PATH_INFO: `/v3/users/af65cb1aeef849d090ecd01f4507392c` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:47.354215 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c37565-5e21-4db8-ad79-9279700d0262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af65cb1aeef849d090ecd01f4507392c)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:47.356241 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c37565-5e21-4db8-ad79-9279700d0262 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:47.435970 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-91c37565-5e21-4db8-ad79-9279700d0262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af65cb1aeef849d090ecd01f4507392c'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:47.440087 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-91c37565-5e21-4db8-ad79-9279700d0262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af65cb1aeef849d090ecd01f4507392c'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:47.443647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-91c37565-5e21-4db8-ad79-9279700d0262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af65cb1aeef849d090ecd01f4507392c'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:47.460459 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-91c37565-5e21-4db8-ad79-9279700d0262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af65cb1aeef849d090ecd01f4507392c'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:47.483068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-91c37565-5e21-4db8-ad79-9279700d0262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af65cb1aeef849d090ecd01f4507392c'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:47.521924 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1352/2687] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:30:47 2022] DELETE /identity/v3/users/af65cb1aeef849d090ecd01f4507392c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:47.944250 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b8a5a-4b2c-4adb-a5df-c887c364f968 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:47.947061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-050b8a5a-4b2c-4adb-a5df-c887c364f968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:47.965438 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b8a5a-4b2c-4adb-a5df-c887c364f968 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:48.013258 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:48.039497 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e9903-d282-4e67-a23f-cb2b9d7f266e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:48.040277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e9903-d282-4e67-a23f-cb2b9d7f266e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:48.040628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e9903-d282-4e67-a23f-cb2b9d7f266e admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:48.040972 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e9903-d282-4e67-a23f-cb2b9d7f266e admin admin] PATH_INFO: `/v3/projects/20944d9e8b5d41f8a8c8cb402660820a` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:48.042203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e9903-d282-4e67-a23f-cb2b9d7f266e admin admin] RBAC: Authorizing `identity:delete_project(project_id=20944d9e8b5d41f8a8c8cb402660820a)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:48.044320 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e9903-d282-4e67-a23f-cb2b9d7f266e admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:48.092897 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-970e9903-d282-4e67-a23f-cb2b9d7f266e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20944d9e8b5d41f8a8c8cb402660820a', 'request_id': 'req-970e9903-d282-4e67-a23f-cb2b9d7f266e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:48.099256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-970e9903-d282-4e67-a23f-cb2b9d7f266e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20944d9e8b5d41f8a8c8cb402660820a', 'request_id': 'req-970e9903-d282-4e67-a23f-cb2b9d7f266e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:48.104832 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-970e9903-d282-4e67-a23f-cb2b9d7f266e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20944d9e8b5d41f8a8c8cb402660820a', 'request_id': 'req-970e9903-d282-4e67-a23f-cb2b9d7f266e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:48.117332 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-970e9903-d282-4e67-a23f-cb2b9d7f266e admin admin] The token cache is being invalidate because project 20944d9e8b5d41f8a8c8cb402660820a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:30:48.117566 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-970e9903-d282-4e67-a23f-cb2b9d7f266e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:48.118174 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-970e9903-d282-4e67-a23f-cb2b9d7f266e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:48.119643 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1336/2688] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:30:47 2022] DELETE /identity/v3/projects/20944d9e8b5d41f8a8c8cb402660820a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:48.125039 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e960d25-c676-46d6-a1d8-caa17a0ea813 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:48.125326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e960d25-c676-46d6-a1d8-caa17a0ea813 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:48.125597 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e960d25-c676-46d6-a1d8-caa17a0ea813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:48.145598 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-2e960d25-c676-46d6-a1d8-caa17a0ea813 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:48.146784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2e960d25-c676-46d6-a1d8-caa17a0ea813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:48.204819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1353/2689] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:48.212660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1da363-1950-4d7e-8de3-58a6e13c624d None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:48.212947 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1da363-1950-4d7e-8de3-58a6e13c624d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:48.213220 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1da363-1950-4d7e-8de3-58a6e13c624d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:48.233406 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-7b1da363-1950-4d7e-8de3-58a6e13c624d None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:48.234640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7b1da363-1950-4d7e-8de3-58a6e13c624d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:48.243755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1337/2690] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:48.248420 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e960d25-c676-46d6-a1d8-caa17a0ea813 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:48.259350 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e960d25-c676-46d6-a1d8-caa17a0ea813 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:48.265821 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d377f51-8948-4e8c-b73b-560dc1b03a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:48.266430 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d377f51-8948-4e8c-b73b-560dc1b03a2f admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:48.266730 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d377f51-8948-4e8c-b73b-560dc1b03a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:48.267065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d377f51-8948-4e8c-b73b-560dc1b03a2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:48.267757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d377f51-8948-4e8c-b73b-560dc1b03a2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:30:48.268106 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d377f51-8948-4e8c-b73b-560dc1b03a2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:48.271426 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d377f51-8948-4e8c-b73b-560dc1b03a2f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:48.274263 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:48.289232 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1354/2691] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:30:48.293432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1da363-1950-4d7e-8de3-58a6e13c624d None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:48.304135 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1da363-1950-4d7e-8de3-58a6e13c624d None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:48.309901 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22411e35-b622-44a8-8d4f-01c51ffc3bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:48.310513 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22411e35-b622-44a8-8d4f-01c51ffc3bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:48.310796 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22411e35-b622-44a8-8d4f-01c51ffc3bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:48.311068 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22411e35-b622-44a8-8d4f-01c51ffc3bb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:48.311720 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22411e35-b622-44a8-8d4f-01c51ffc3bb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:48.313659 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22411e35-b622-44a8-8d4f-01c51ffc3bb8 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:48.355531 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1338/2692] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:30:48 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:30:48.356817 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d377f51-8948-4e8c-b73b-560dc1b03a2f admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:48.369722 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d377f51-8948-4e8c-b73b-560dc1b03a2f admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:48.436526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d1698-e052-4294-984c-b2264b3c2e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:48.437371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d1698-e052-4294-984c-b2264b3c2e8f admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:48.437664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d1698-e052-4294-984c-b2264b3c2e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:48.438443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d1698-e052-4294-984c-b2264b3c2e8f admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:48.438571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d1698-e052-4294-984c-b2264b3c2e8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:48.440604 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d1698-e052-4294-984c-b2264b3c2e8f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:48.467906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1355/2693] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:30:48 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:30:48.469175 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22411e35-b622-44a8-8d4f-01c51ffc3bb8 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:48.485096 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22411e35-b622-44a8-8d4f-01c51ffc3bb8 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:48.500084 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066256b0-65eb-4497-bf34-6f51cab3c896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:48.500844 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066256b0-65eb-4497-bf34-6f51cab3c896 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:48.501177 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066256b0-65eb-4497-bf34-6f51cab3c896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:48.501450 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066256b0-65eb-4497-bf34-6f51cab3c896 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:48.502257 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066256b0-65eb-4497-bf34-6f51cab3c896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:48.504486 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066256b0-65eb-4497-bf34-6f51cab3c896 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:48.515253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1339/2694] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:30:48.520329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d1698-e052-4294-984c-b2264b3c2e8f admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:48.532057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d1698-e052-4294-984c-b2264b3c2e8f admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:48.536732 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b3b3f-5189-47ad-94cf-9ef46e0a5821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:48.537362 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b3b3f-5189-47ad-94cf-9ef46e0a5821 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:48.537637 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b3b3f-5189-47ad-94cf-9ef46e0a5821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:48.537906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b3b3f-5189-47ad-94cf-9ef46e0a5821 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:48.538546 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b3b3f-5189-47ad-94cf-9ef46e0a5821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:48.540368 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b3b3f-5189-47ad-94cf-9ef46e0a5821 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:48.552300 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1356/2695] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:48.556603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066256b0-65eb-4497-bf34-6f51cab3c896 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:48.567734 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066256b0-65eb-4497-bf34-6f51cab3c896 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:48.572425 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864abe5-8ed3-49f1-883c-34b2ed145893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:48.573138 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864abe5-8ed3-49f1-883c-34b2ed145893 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:48.573483 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864abe5-8ed3-49f1-883c-34b2ed145893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:48.573843 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864abe5-8ed3-49f1-883c-34b2ed145893 admin admin] PATH_INFO: `/v3/projects/64bbe5d1e1494ae5acdc51510e5c859b/users/5b3ff5cbf5d547bf80c2b68a5b6bcde2/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:48.589063 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864abe5-8ed3-49f1-883c-34b2ed145893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64bbe5d1e1494ae5acdc51510e5c859b, user_id=5b3ff5cbf5d547bf80c2b68a5b6bcde2, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:48.591850 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864abe5-8ed3-49f1-883c-34b2ed145893 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:48.604885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1340/2696] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:48 2022] PUT /identity/v3/projects/64bbe5d1e1494ae5acdc51510e5c859b/users/5b3ff5cbf5d547bf80c2b68a5b6bcde2/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:48.617616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f195fcf-d232-4da7-ba25-8e8ff917d1b9 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:48.617971 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f195fcf-d232-4da7-ba25-8e8ff917d1b9 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:48.618279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f195fcf-d232-4da7-ba25-8e8ff917d1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:48.650024 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-5f195fcf-d232-4da7-ba25-8e8ff917d1b9 None None] MFA Rules not processed for user `5b3ff5cbf5d547bf80c2b68a5b6bcde2`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:48.651358 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5f195fcf-d232-4da7-ba25-8e8ff917d1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:48.757531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1357/2697] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:48.761854 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864abe5-8ed3-49f1-883c-34b2ed145893 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:48.834054 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864abe5-8ed3-49f1-883c-34b2ed145893 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:48.839800 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9373fc2-466c-483a-8e48-0be5f9f0a43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:48.840280 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9373fc2-466c-483a-8e48-0be5f9f0a43d admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:48.841172 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9373fc2-466c-483a-8e48-0be5f9f0a43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:48.841512 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9373fc2-466c-483a-8e48-0be5f9f0a43d admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:48.842306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9373fc2-466c-483a-8e48-0be5f9f0a43d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:48.844369 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9373fc2-466c-483a-8e48-0be5f9f0a43d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:48.854597 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:48.880079 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1341/2698] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:30:48 2022] POST /identity/v3/projects => generated 381 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:30:48.884002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f195fcf-d232-4da7-ba25-8e8ff917d1b9 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:48.896966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f195fcf-d232-4da7-ba25-8e8ff917d1b9 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:48.977421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73cd9b1-9355-4d06-a01f-04346220edf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:48.978057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cd9b1-9355-4d06-a01f-04346220edf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:48.978338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cd9b1-9355-4d06-a01f-04346220edf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:48.978608 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cd9b1-9355-4d06-a01f-04346220edf0 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:48.979304 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73cd9b1-9355-4d06-a01f-04346220edf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:48.982100 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73cd9b1-9355-4d06-a01f-04346220edf0 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:48.991862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:49.008045 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1358/2699] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:30:48 2022] POST /identity/v3/users => generated 291 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:30:49.011921 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9373fc2-466c-483a-8e48-0be5f9f0a43d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:49.023948 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9373fc2-466c-483a-8e48-0be5f9f0a43d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:49.030103 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82cd07e-1403-4d30-91ce-a120e80687ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:49.030784 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cd07e-1403-4d30-91ce-a120e80687ba admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:49.031142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cd07e-1403-4d30-91ce-a120e80687ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:49.031502 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cd07e-1403-4d30-91ce-a120e80687ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:49.032254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82cd07e-1403-4d30-91ce-a120e80687ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:49.033833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82cd07e-1403-4d30-91ce-a120e80687ba admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:49.051348 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73cd9b1-9355-4d06-a01f-04346220edf0 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:49.051580 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1342/2700] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:30:49.071351 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73cd9b1-9355-4d06-a01f-04346220edf0 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:49.080879 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5447d5-338c-4d91-8597-e1d23b296964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:49.082018 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5447d5-338c-4d91-8597-e1d23b296964 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:49.082576 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5447d5-338c-4d91-8597-e1d23b296964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:49.082862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5447d5-338c-4d91-8597-e1d23b296964 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:49.083585 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5447d5-338c-4d91-8597-e1d23b296964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:49.087707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5447d5-338c-4d91-8597-e1d23b296964 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:49.103392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1359/2701] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:49.108056 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82cd07e-1403-4d30-91ce-a120e80687ba admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:49.122602 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82cd07e-1403-4d30-91ce-a120e80687ba admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:49.129794 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cde37-0be8-49ce-999d-6fb7884fa530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:49.130404 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cde37-0be8-49ce-999d-6fb7884fa530 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:49.130814 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cde37-0be8-49ce-999d-6fb7884fa530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:49.132478 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cde37-0be8-49ce-999d-6fb7884fa530 admin admin] PATH_INFO: `/v3/projects/5c2632354691463a8d846796f31eb610/users/92272aea13944ebabf4d2962f5d2c464/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:49.154388 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9cde37-0be8-49ce-999d-6fb7884fa530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c2632354691463a8d846796f31eb610, user_id=92272aea13944ebabf4d2962f5d2c464, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:49.158014 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9cde37-0be8-49ce-999d-6fb7884fa530 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:49.174368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1343/2702] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:49 2022] PUT /identity/v3/projects/5c2632354691463a8d846796f31eb610/users/92272aea13944ebabf4d2962f5d2c464/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:49.185063 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0d30c-b41f-46e8-916c-09b6950a30a7 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:49.185063 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0d30c-b41f-46e8-916c-09b6950a30a7 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:49.185349 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0d30c-b41f-46e8-916c-09b6950a30a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:49.218038 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-59e0d30c-b41f-46e8-916c-09b6950a30a7 None None] MFA Rules not processed for user `92272aea13944ebabf4d2962f5d2c464`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:49.220256 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-59e0d30c-b41f-46e8-916c-09b6950a30a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:49.326276 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1360/2703] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:49.329530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cde37-0be8-49ce-999d-6fb7884fa530 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:49.412878 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cde37-0be8-49ce-999d-6fb7884fa530 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:49.417601 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a701c8-ce08-4af0-a0d5-910209e51d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:49.418229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a701c8-ce08-4af0-a0d5-910209e51d6b admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:49.419052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a701c8-ce08-4af0-a0d5-910209e51d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:49.419327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a701c8-ce08-4af0-a0d5-910209e51d6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:49.419986 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a701c8-ce08-4af0-a0d5-910209e51d6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:49.421890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a701c8-ce08-4af0-a0d5-910209e51d6b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:49.450268 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1344/2704] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:30:49 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:30:49.454383 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0d30c-b41f-46e8-916c-09b6950a30a7 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:49.466217 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0d30c-b41f-46e8-916c-09b6950a30a7 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:49.534198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9844824-8365-4685-8897-f1af56b44f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:49.534927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9844824-8365-4685-8897-f1af56b44f55 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:49.535212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9844824-8365-4685-8897-f1af56b44f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:49.535495 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9844824-8365-4685-8897-f1af56b44f55 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:49.536350 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9844824-8365-4685-8897-f1af56b44f55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:49.538324 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9844824-8365-4685-8897-f1af56b44f55 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:49.568482 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1361/2705] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:30:49 2022] POST /identity/v3/users => generated 299 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:30:49.574489 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a701c8-ce08-4af0-a0d5-910209e51d6b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:49.587492 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a701c8-ce08-4af0-a0d5-910209e51d6b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:49.593159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ac079-9206-40ba-a61e-163a5c348105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:49.593844 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ac079-9206-40ba-a61e-163a5c348105 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:49.594193 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ac079-9206-40ba-a61e-163a5c348105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:49.594675 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ac079-9206-40ba-a61e-163a5c348105 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:49.595398 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ac079-9206-40ba-a61e-163a5c348105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:49.597374 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ac079-9206-40ba-a61e-163a5c348105 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:49.610421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1345/2706] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:49.614884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9844824-8365-4685-8897-f1af56b44f55 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:49.626695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9844824-8365-4685-8897-f1af56b44f55 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:49.632805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd3a61-b93e-4cda-9b39-44e9ded4f13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:49.633791 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd3a61-b93e-4cda-9b39-44e9ded4f13b admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:49.634008 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd3a61-b93e-4cda-9b39-44e9ded4f13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:49.634841 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd3a61-b93e-4cda-9b39-44e9ded4f13b admin admin] PATH_INFO: `/v3/projects/ea577858f8ad425c958f1cdbecbdc934/users/29893f5593c249769ad35111ac8aed61/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:49.666707 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd3a61-b93e-4cda-9b39-44e9ded4f13b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea577858f8ad425c958f1cdbecbdc934, user_id=29893f5593c249769ad35111ac8aed61, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:49.671542 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd3a61-b93e-4cda-9b39-44e9ded4f13b admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:49.693574 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1362/2707] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:49 2022] PUT /identity/v3/projects/ea577858f8ad425c958f1cdbecbdc934/users/29893f5593c249769ad35111ac8aed61/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:49.697378 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ac079-9206-40ba-a61e-163a5c348105 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:49.708872 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ac079-9206-40ba-a61e-163a5c348105 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:49.770409 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22509894-aa9e-4712-b17a-148c61a89e28 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:49.770726 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22509894-aa9e-4712-b17a-148c61a89e28 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:49.771002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22509894-aa9e-4712-b17a-148c61a89e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:49.802137 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-22509894-aa9e-4712-b17a-148c61a89e28 None None] MFA Rules not processed for user `06d279b488d148428b7a3c35a9786ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:49.803349 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-22509894-aa9e-4712-b17a-148c61a89e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:49.810058 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fe272-abc2-4ea8-9d41-eb0d3135ba45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:49.810681 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fe272-abc2-4ea8-9d41-eb0d3135ba45 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:49.811468 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fe272-abc2-4ea8-9d41-eb0d3135ba45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:49.812051 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fe272-abc2-4ea8-9d41-eb0d3135ba45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:49.812707 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92fe272-abc2-4ea8-9d41-eb0d3135ba45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:49.814814 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92fe272-abc2-4ea8-9d41-eb0d3135ba45 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:49.825781 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1346/2708] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:30:49.829816 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fe272-abc2-4ea8-9d41-eb0d3135ba45 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:49.841858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fe272-abc2-4ea8-9d41-eb0d3135ba45 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:49.847946 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d134a848-80e3-4d51-8ab1-1cd623233330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:49.848655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134a848-80e3-4d51-8ab1-1cd623233330 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:49.848957 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134a848-80e3-4d51-8ab1-1cd623233330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:49.849086 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:49.849256 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134a848-80e3-4d51-8ab1-1cd623233330 admin admin] PATH_INFO: `/v3/projects/ea577858f8ad425c958f1cdbecbdc934/users/29893f5593c249769ad35111ac8aed61/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:49.851534 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d134a848-80e3-4d51-8ab1-1cd623233330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea577858f8ad425c958f1cdbecbdc934, user_id=29893f5593c249769ad35111ac8aed61, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:49.854787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d134a848-80e3-4d51-8ab1-1cd623233330 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:49.866228 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1347/2709] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:30:49 2022] PUT /identity/v3/projects/ea577858f8ad425c958f1cdbecbdc934/users/29893f5593c249769ad35111ac8aed61/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:49.877915 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1363/2710] 10.222.0.81 () {58 vars in 1009 bytes} [Mon Sep 19 11:30:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:49.879673 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce38894-c0e7-43e6-ad75-3384d36b59bb None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:49.880001 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce38894-c0e7-43e6-ad75-3384d36b59bb None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:49.880275 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce38894-c0e7-43e6-ad75-3384d36b59bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:49.902885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:49.912553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-bce38894-c0e7-43e6-ad75-3384d36b59bb None None] MFA Rules not processed for user `29893f5593c249769ad35111ac8aed61`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:49.914012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bce38894-c0e7-43e6-ad75-3384d36b59bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:49.934477 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22509894-aa9e-4712-b17a-148c61a89e28 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:49.935993 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-22509894-aa9e-4712-b17a-148c61a89e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:49.986581 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1348/2711] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:30:49.998000 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce38894-c0e7-43e6-ad75-3384d36b59bb None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:49.999406 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bce38894-c0e7-43e6-ad75-3384d36b59bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:50.006870 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22509894-aa9e-4712-b17a-148c61a89e28 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:50.062930 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce38894-c0e7-43e6-ad75-3384d36b59bb None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:50.068899 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e77e9-edee-4e33-8011-e096528a5a62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:50.069508 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e77e9-edee-4e33-8011-e096528a5a62 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:50.069898 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e77e9-edee-4e33-8011-e096528a5a62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:50.070208 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e77e9-edee-4e33-8011-e096528a5a62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:50.073900 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ab39e-aed4-4b96-8a1a-0431f5ea654f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8a9732807d4921a41a6a2d5019e2f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:50.074616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ab39e-aed4-4b96-8a1a-0431f5ea654f service swift] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:50.075998 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ab39e-aed4-4b96-8a1a-0431f5ea654f service swift] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:50.076278 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ab39e-aed4-4b96-8a1a-0431f5ea654f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:50.142468 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e77e9-edee-4e33-8011-e096528a5a62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:50.145301 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e77e9-edee-4e33-8011-e096528a5a62 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:50.147063 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ab39e-aed4-4b96-8a1a-0431f5ea654f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:50.148759 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ab39e-aed4-4b96-8a1a-0431f5ea654f service swift] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:50.160264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1364/2713] 10.222.0.81 () {60 vars in 1384 bytes} [Mon Sep 19 11:30:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 11:30:50.166747 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1349/2713] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:30:50.376634 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ab39e-aed4-4b96-8a1a-0431f5ea654f service swift] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:50.378274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5e0ab39e-aed4-4b96-8a1a-0431f5ea654f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:50.417475 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:50.467941 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ab39e-aed4-4b96-8a1a-0431f5ea654f service swift] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:50.472867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1d8f9-b0d1-4114-806f-f72958499b8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:50.473728 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1d8f9-b0d1-4114-806f-f72958499b8e service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:50.474288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1d8f9-b0d1-4114-806f-f72958499b8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:50.474751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1d8f9-b0d1-4114-806f-f72958499b8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:50.477995 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-31a1d8f9-b0d1-4114-806f-f72958499b8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:50.597831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1d8f9-b0d1-4114-806f-f72958499b8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:50.600366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1d8f9-b0d1-4114-806f-f72958499b8e service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:50.612661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1365/2714] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:50.823961 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a4e2c-9899-466b-9523-aa87615ffb0b None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:50.824325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a4e2c-9899-466b-9523-aa87615ffb0b None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:50.825199 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a4e2c-9899-466b-9523-aa87615ffb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:50.847099 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-fa8a4e2c-9899-466b-9523-aa87615ffb0b None None] MFA Rules not processed for user `06d279b488d148428b7a3c35a9786ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:50.848280 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-fa8a4e2c-9899-466b-9523-aa87615ffb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:50.860503 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1350/2715] 10.222.0.81 () {58 vars in 967 bytes} [Mon Sep 19 11:30:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:30:50.873968 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1d8f9-b0d1-4114-806f-f72958499b8e service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:50.887711 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1d8f9-b0d1-4114-806f-f72958499b8e service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:50.894513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814d81a-4bc8-4557-9cda-262673af58e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8a9732807d4921a41a6a2d5019e2f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:50.895154 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814d81a-4bc8-4557-9cda-262673af58e4 service swift] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:50.895637 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814d81a-4bc8-4557-9cda-262673af58e4 service swift] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:50.895878 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814d81a-4bc8-4557-9cda-262673af58e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:50.906340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814d81a-4bc8-4557-9cda-262673af58e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:50.908466 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814d81a-4bc8-4557-9cda-262673af58e4 service swift] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:50.919743 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1366/2716] 10.222.0.81 () {60 vars in 1384 bytes} [Mon Sep 19 11:30:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:30:51.175600 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e7491-7829-4b0d-8c38-b84ff1d3e470 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:51.176188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e7491-7829-4b0d-8c38-b84ff1d3e470 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:51.176188 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e7491-7829-4b0d-8c38-b84ff1d3e470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:51.196775 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-fc3e7491-7829-4b0d-8c38-b84ff1d3e470 None None] MFA Rules not processed for user `06d279b488d148428b7a3c35a9786ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:51.197994 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-fc3e7491-7829-4b0d-8c38-b84ff1d3e470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:51.206213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1351/2717] 10.222.0.81 () {58 vars in 1009 bytes} [Mon Sep 19 11:30:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:51.218988 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814d81a-4bc8-4557-9cda-262673af58e4 service swift] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:51.229937 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814d81a-4bc8-4557-9cda-262673af58e4 service swift] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:51.235807 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142f487-2108-41f3-af16-8da77fa6c3be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8a9732807d4921a41a6a2d5019e2f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:51.236411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142f487-2108-41f3-af16-8da77fa6c3be service swift] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:51.236687 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142f487-2108-41f3-af16-8da77fa6c3be service swift] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:51.236957 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142f487-2108-41f3-af16-8da77fa6c3be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:51.245616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142f487-2108-41f3-af16-8da77fa6c3be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:51.247204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142f487-2108-41f3-af16-8da77fa6c3be service swift] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:51.255560 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1367/2718] 10.222.0.81 () {60 vars in 1384 bytes} [Mon Sep 19 11:30:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:30:51.610608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e74dc-8211-4017-a9bc-fdb678c1dc41 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:51.610902 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e74dc-8211-4017-a9bc-fdb678c1dc41 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:51.611176 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e74dc-8211-4017-a9bc-fdb678c1dc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:51.633607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-489e74dc-8211-4017-a9bc-fdb678c1dc41 None None] MFA Rules not processed for user `06d279b488d148428b7a3c35a9786ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:51.634920 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-489e74dc-8211-4017-a9bc-fdb678c1dc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:51.644219 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1352/2719] 10.222.0.81 () {58 vars in 967 bytes} [Mon Sep 19 11:30:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:51.661327 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142f487-2108-41f3-af16-8da77fa6c3be service swift] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:51.675003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142f487-2108-41f3-af16-8da77fa6c3be service swift] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:51.682156 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b427b5-cac8-4eed-a0f0-6667171b6486 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8a9732807d4921a41a6a2d5019e2f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:51.682770 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b427b5-cac8-4eed-a0f0-6667171b6486 service swift] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:51.683264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b427b5-cac8-4eed-a0f0-6667171b6486 service swift] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:51.683607 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b427b5-cac8-4eed-a0f0-6667171b6486 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:51.694077 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b427b5-cac8-4eed-a0f0-6667171b6486 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:51.695745 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b427b5-cac8-4eed-a0f0-6667171b6486 service swift] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:51.705195 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1368/2720] 10.222.0.81 () {60 vars in 1384 bytes} [Mon Sep 19 11:30:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 11:30:51.854634 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e74dc-8211-4017-a9bc-fdb678c1dc41 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:51.865311 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e74dc-8211-4017-a9bc-fdb678c1dc41 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:51.869800 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e23e6-d79d-4361-8475-04881d12c171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:51.870432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e23e6-d79d-4361-8475-04881d12c171 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:51.870702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e23e6-d79d-4361-8475-04881d12c171 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:51.870968 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e23e6-d79d-4361-8475-04881d12c171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:51.904299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=100 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:51.948823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e23e6-d79d-4361-8475-04881d12c171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:51.950193 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e23e6-d79d-4361-8475-04881d12c171 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:51.977400 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1353/2721] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:52.743545 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b427b5-cac8-4eed-a0f0-6667171b6486 service swift] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:52.746041 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a9b427b5-cac8-4eed-a0f0-6667171b6486 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:52.759274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b427b5-cac8-4eed-a0f0-6667171b6486 service swift] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:52.763122 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41146b88-236c-4ab6-8bb1-1fb7c660e0bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:52.767967 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41146b88-236c-4ab6-8bb1-1fb7c660e0bf service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:52.768270 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41146b88-236c-4ab6-8bb1-1fb7c660e0bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:52.768805 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41146b88-236c-4ab6-8bb1-1fb7c660e0bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:52.780585 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41146b88-236c-4ab6-8bb1-1fb7c660e0bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:52.782032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41146b88-236c-4ab6-8bb1-1fb7c660e0bf service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:52.799200 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1369/2722] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:54.126893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e23e6-d79d-4361-8475-04881d12c171 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:54.129504 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d46e23e6-d79d-4361-8475-04881d12c171 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:54.155864 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e23e6-d79d-4361-8475-04881d12c171 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:54.162245 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021ff16-2758-4e80-bd43-96a66bc1c397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:54.163317 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021ff16-2758-4e80-bd43-96a66bc1c397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:54.163940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021ff16-2758-4e80-bd43-96a66bc1c397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:54.164472 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021ff16-2758-4e80-bd43-96a66bc1c397 admin admin] PATH_INFO: `/v3/users/7dbae40715f9423b82b1ee34ef30f5ec` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:54.166260 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9021ff16-2758-4e80-bd43-96a66bc1c397 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dbae40715f9423b82b1ee34ef30f5ec)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:54.169236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9021ff16-2758-4e80-bd43-96a66bc1c397 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:54.292863 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9021ff16-2758-4e80-bd43-96a66bc1c397 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dbae40715f9423b82b1ee34ef30f5ec'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:54.304298 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9021ff16-2758-4e80-bd43-96a66bc1c397 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dbae40715f9423b82b1ee34ef30f5ec'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:54.305422 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9021ff16-2758-4e80-bd43-96a66bc1c397 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dbae40715f9423b82b1ee34ef30f5ec'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:54.306410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9021ff16-2758-4e80-bd43-96a66bc1c397 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dbae40715f9423b82b1ee34ef30f5ec'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:54.323785 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-9021ff16-2758-4e80-bd43-96a66bc1c397 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dbae40715f9423b82b1ee34ef30f5ec'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:54.345120 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1354/2723] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:30:54 2022] DELETE /identity/v3/users/7dbae40715f9423b82b1ee34ef30f5ec => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:54.349447 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fbbb4-56b5-41e9-b87c-039a25ce7083 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:54.349945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fbbb4-56b5-41e9-b87c-039a25ce7083 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:54.350226 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fbbb4-56b5-41e9-b87c-039a25ce7083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:54.383998 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-b37fbbb4-56b5-41e9-b87c-039a25ce7083 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:54.385411 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b37fbbb4-56b5-41e9-b87c-039a25ce7083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:54.458765 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1370/2724] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:54.468085 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021ff16-2758-4e80-bd43-96a66bc1c397 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:54.469962 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9021ff16-2758-4e80-bd43-96a66bc1c397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:54.537607 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021ff16-2758-4e80-bd43-96a66bc1c397 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:54.541606 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780188a-03e4-4146-9ff9-f5a8db9a5791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:54.542282 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780188a-03e4-4146-9ff9-f5a8db9a5791 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:54.542671 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780188a-03e4-4146-9ff9-f5a8db9a5791 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:54.543007 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780188a-03e4-4146-9ff9-f5a8db9a5791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:54.554033 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3780188a-03e4-4146-9ff9-f5a8db9a5791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:54.555646 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3780188a-03e4-4146-9ff9-f5a8db9a5791 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:54.567913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1355/2725] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:54.850203 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fbbb4-56b5-41e9-b87c-039a25ce7083 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:54.852218 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b37fbbb4-56b5-41e9-b87c-039a25ce7083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:54.870403 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fbbb4-56b5-41e9-b87c-039a25ce7083 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:54.875268 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740b552-04c5-4972-93bf-cc822f49b3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:54.876613 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740b552-04c5-4972-93bf-cc822f49b3bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:54.877557 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740b552-04c5-4972-93bf-cc822f49b3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:54.877841 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740b552-04c5-4972-93bf-cc822f49b3bd admin admin] PATH_INFO: `/v3/projects/f446bc44fa074e80b2b477bdf13bcdcf` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:54.879663 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740b552-04c5-4972-93bf-cc822f49b3bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f446bc44fa074e80b2b477bdf13bcdcf)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:54.882484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740b552-04c5-4972-93bf-cc822f49b3bd admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:54.906160 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:54.941277 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-0740b552-04c5-4972-93bf-cc822f49b3bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f446bc44fa074e80b2b477bdf13bcdcf', 'request_id': 'req-0740b552-04c5-4972-93bf-cc822f49b3bd'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:54.959298 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-0740b552-04c5-4972-93bf-cc822f49b3bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f446bc44fa074e80b2b477bdf13bcdcf', 'request_id': 'req-0740b552-04c5-4972-93bf-cc822f49b3bd'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:54.967957 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-0740b552-04c5-4972-93bf-cc822f49b3bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f446bc44fa074e80b2b477bdf13bcdcf', 'request_id': 'req-0740b552-04c5-4972-93bf-cc822f49b3bd'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:54.991947 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-0740b552-04c5-4972-93bf-cc822f49b3bd admin admin] The token cache is being invalidate because project f446bc44fa074e80b2b477bdf13bcdcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:30:54.992248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-0740b552-04c5-4972-93bf-cc822f49b3bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:54.993030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-0740b552-04c5-4972-93bf-cc822f49b3bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:54.999895 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1371/2726] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:30:54 2022] DELETE /identity/v3/projects/f446bc44fa074e80b2b477bdf13bcdcf => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:57.001183 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780188a-03e4-4146-9ff9-f5a8db9a5791 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:57.004086 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3780188a-03e4-4146-9ff9-f5a8db9a5791 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:57.020821 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780188a-03e4-4146-9ff9-f5a8db9a5791 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:57.081480 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:57.108050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea7c5d-464f-44fe-89c4-7273231904cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:57.108972 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea7c5d-464f-44fe-89c4-7273231904cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:57.109413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea7c5d-464f-44fe-89c4-7273231904cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:57.109855 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea7c5d-464f-44fe-89c4-7273231904cc admin admin] PATH_INFO: `/v3/users/5b3ff5cbf5d547bf80c2b68a5b6bcde2` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:57.111274 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea7c5d-464f-44fe-89c4-7273231904cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b3ff5cbf5d547bf80c2b68a5b6bcde2)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:57.113551 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea7c5d-464f-44fe-89c4-7273231904cc admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:57.169662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-44ea7c5d-464f-44fe-89c4-7273231904cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b3ff5cbf5d547bf80c2b68a5b6bcde2'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:57.177555 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-44ea7c5d-464f-44fe-89c4-7273231904cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b3ff5cbf5d547bf80c2b68a5b6bcde2'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:57.178496 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-44ea7c5d-464f-44fe-89c4-7273231904cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b3ff5cbf5d547bf80c2b68a5b6bcde2'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:57.179134 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-44ea7c5d-464f-44fe-89c4-7273231904cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b3ff5cbf5d547bf80c2b68a5b6bcde2'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:57.197625 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-44ea7c5d-464f-44fe-89c4-7273231904cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b3ff5cbf5d547bf80c2b68a5b6bcde2'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:57.215872 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1356/2727] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:30:56 2022] DELETE /identity/v3/users/5b3ff5cbf5d547bf80c2b68a5b6bcde2 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:57.220461 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ae113-3633-4568-88a8-2bda5d1ebf99 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:57.221317 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ae113-3633-4568-88a8-2bda5d1ebf99 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:57.221660 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ae113-3633-4568-88a8-2bda5d1ebf99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:57.242215 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-182ae113-3633-4568-88a8-2bda5d1ebf99 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:57.243528 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-182ae113-3633-4568-88a8-2bda5d1ebf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:57.313056 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1372/2728] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:30:57.323420 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea7c5d-464f-44fe-89c4-7273231904cc admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:57.325828 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-44ea7c5d-464f-44fe-89c4-7273231904cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:57.387730 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea7c5d-464f-44fe-89c4-7273231904cc admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:57.392064 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63cf07-d0fb-474d-8f3a-02d7e529b540 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:57.392912 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63cf07-d0fb-474d-8f3a-02d7e529b540 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:57.393540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63cf07-d0fb-474d-8f3a-02d7e529b540 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:57.393922 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63cf07-d0fb-474d-8f3a-02d7e529b540 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:57.407231 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d63cf07-d0fb-474d-8f3a-02d7e529b540 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:57.408856 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d63cf07-d0fb-474d-8f3a-02d7e529b540 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:57.420276 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1357/2729] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:30:57.802757 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ae113-3633-4568-88a8-2bda5d1ebf99 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:57.804252 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-182ae113-3633-4568-88a8-2bda5d1ebf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:57.829351 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ae113-3633-4568-88a8-2bda5d1ebf99 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:57.839212 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd512c-ee94-4386-ac8b-41b1e2a5a199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:57.841705 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd512c-ee94-4386-ac8b-41b1e2a5a199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:57.842487 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd512c-ee94-4386-ac8b-41b1e2a5a199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:57.851471 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd512c-ee94-4386-ac8b-41b1e2a5a199 admin admin] PATH_INFO: `/v3/projects/64bbe5d1e1494ae5acdc51510e5c859b` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:57.852924 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd512c-ee94-4386-ac8b-41b1e2a5a199 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64bbe5d1e1494ae5acdc51510e5c859b)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:57.854879 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd512c-ee94-4386-ac8b-41b1e2a5a199 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:57.912946 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8bbd512c-ee94-4386-ac8b-41b1e2a5a199 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64bbe5d1e1494ae5acdc51510e5c859b', 'request_id': 'req-8bbd512c-ee94-4386-ac8b-41b1e2a5a199'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:57.920022 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8bbd512c-ee94-4386-ac8b-41b1e2a5a199 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64bbe5d1e1494ae5acdc51510e5c859b', 'request_id': 'req-8bbd512c-ee94-4386-ac8b-41b1e2a5a199'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:57.927052 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8bbd512c-ee94-4386-ac8b-41b1e2a5a199 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64bbe5d1e1494ae5acdc51510e5c859b', 'request_id': 'req-8bbd512c-ee94-4386-ac8b-41b1e2a5a199'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:57.941374 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8bbd512c-ee94-4386-ac8b-41b1e2a5a199 admin admin] The token cache is being invalidate because project 64bbe5d1e1494ae5acdc51510e5c859b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:30:57.941663 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8bbd512c-ee94-4386-ac8b-41b1e2a5a199 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:57.942495 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-8bbd512c-ee94-4386-ac8b-41b1e2a5a199 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:57.944517 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1373/2730] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:30:57 2022] DELETE /identity/v3/projects/64bbe5d1e1494ae5acdc51510e5c859b => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:57.948671 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63cf07-d0fb-474d-8f3a-02d7e529b540 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:57.965152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63cf07-d0fb-474d-8f3a-02d7e529b540 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:58.040340 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab81ac2-d452-4c16-a3ac-469559fd485b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:58.041060 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab81ac2-d452-4c16-a3ac-469559fd485b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:58.041250 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab81ac2-d452-4c16-a3ac-469559fd485b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:58.041722 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab81ac2-d452-4c16-a3ac-469559fd485b admin admin] PATH_INFO: `/v3/users/92272aea13944ebabf4d2962f5d2c464` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:58.042895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab81ac2-d452-4c16-a3ac-469559fd485b admin admin] RBAC: Authorizing `identity:delete_user(user_id=92272aea13944ebabf4d2962f5d2c464)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:58.044909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab81ac2-d452-4c16-a3ac-469559fd485b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:58.052486 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:58.105530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7ab81ac2-d452-4c16-a3ac-469559fd485b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92272aea13944ebabf4d2962f5d2c464'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:58.117768 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7ab81ac2-d452-4c16-a3ac-469559fd485b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92272aea13944ebabf4d2962f5d2c464'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:58.118695 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7ab81ac2-d452-4c16-a3ac-469559fd485b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92272aea13944ebabf4d2962f5d2c464'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:58.119164 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7ab81ac2-d452-4c16-a3ac-469559fd485b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92272aea13944ebabf4d2962f5d2c464'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:58.136330 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-7ab81ac2-d452-4c16-a3ac-469559fd485b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92272aea13944ebabf4d2962f5d2c464'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:58.154042 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1358/2731] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:30:57 2022] DELETE /identity/v3/users/92272aea13944ebabf4d2962f5d2c464 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:58.729247 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd512c-ee94-4386-ac8b-41b1e2a5a199 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:58.732080 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-8bbd512c-ee94-4386-ac8b-41b1e2a5a199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:58.745875 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd512c-ee94-4386-ac8b-41b1e2a5a199 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:58.777216 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:58.816896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0912023-5b80-482b-bef0-665466339eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:58.817628 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0912023-5b80-482b-bef0-665466339eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:58.817910 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0912023-5b80-482b-bef0-665466339eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:58.818185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0912023-5b80-482b-bef0-665466339eac admin admin] PATH_INFO: `/v3/projects/5c2632354691463a8d846796f31eb610` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:58.819436 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0912023-5b80-482b-bef0-665466339eac admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c2632354691463a8d846796f31eb610)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:58.821360 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0912023-5b80-482b-bef0-665466339eac admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:58.870323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e0912023-5b80-482b-bef0-665466339eac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c2632354691463a8d846796f31eb610', 'request_id': 'req-e0912023-5b80-482b-bef0-665466339eac'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:58.878201 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e0912023-5b80-482b-bef0-665466339eac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c2632354691463a8d846796f31eb610', 'request_id': 'req-e0912023-5b80-482b-bef0-665466339eac'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:58.884355 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e0912023-5b80-482b-bef0-665466339eac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c2632354691463a8d846796f31eb610', 'request_id': 'req-e0912023-5b80-482b-bef0-665466339eac'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:58.897508 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e0912023-5b80-482b-bef0-665466339eac admin admin] The token cache is being invalidate because project 5c2632354691463a8d846796f31eb610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:30:58.897803 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e0912023-5b80-482b-bef0-665466339eac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:58.898759 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e0912023-5b80-482b-bef0-665466339eac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:58.900567 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1374/2732] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:30:58 2022] DELETE /identity/v3/projects/5c2632354691463a8d846796f31eb610 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:58.904453 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab81ac2-d452-4c16-a3ac-469559fd485b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:58.905858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-7ab81ac2-d452-4c16-a3ac-469559fd485b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:58.919494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab81ac2-d452-4c16-a3ac-469559fd485b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:58.979055 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3101d36-b1b5-4543-aff0-97b5be6906d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:58.979883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3101d36-b1b5-4543-aff0-97b5be6906d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:58.980163 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3101d36-b1b5-4543-aff0-97b5be6906d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:58.980938 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3101d36-b1b5-4543-aff0-97b5be6906d0 admin admin] PATH_INFO: `/v3/users/29893f5593c249769ad35111ac8aed61` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:58.982070 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3101d36-b1b5-4543-aff0-97b5be6906d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29893f5593c249769ad35111ac8aed61)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:58.984031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3101d36-b1b5-4543-aff0-97b5be6906d0 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:59.043551 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a3101d36-b1b5-4543-aff0-97b5be6906d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29893f5593c249769ad35111ac8aed61'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:59.052619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a3101d36-b1b5-4543-aff0-97b5be6906d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29893f5593c249769ad35111ac8aed61'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:59.053673 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a3101d36-b1b5-4543-aff0-97b5be6906d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29893f5593c249769ad35111ac8aed61'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:59.054538 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a3101d36-b1b5-4543-aff0-97b5be6906d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29893f5593c249769ad35111ac8aed61'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:59.070642 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a3101d36-b1b5-4543-aff0-97b5be6906d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29893f5593c249769ad35111ac8aed61'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:59.085029 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1359/2733] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:30:58 2022] DELETE /identity/v3/users/29893f5593c249769ad35111ac8aed61 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:30:59.569740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181802c6-bab0-4a16-ace5-6a77cbb100e9 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:59.570062 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181802c6-bab0-4a16-ace5-6a77cbb100e9 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:59.570374 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181802c6-bab0-4a16-ace5-6a77cbb100e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:59.593347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-181802c6-bab0-4a16-ace5-6a77cbb100e9 None None] MFA Rules not processed for user `c23ac045e3db4798be704a4bf37b3c2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:59.594830 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-181802c6-bab0-4a16-ace5-6a77cbb100e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:59.630489 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3101d36-b1b5-4543-aff0-97b5be6906d0 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:30:59.632168 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a3101d36-b1b5-4543-aff0-97b5be6906d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:59.639155 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:30:59.666705 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1375/2734] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:59.730076 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3101d36-b1b5-4543-aff0-97b5be6906d0 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:30:59.736467 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ff447-6fdc-423b-823c-4e5c23f1a89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:30:59.737293 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ff447-6fdc-423b-823c-4e5c23f1a89d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:59.737672 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ff447-6fdc-423b-823c-4e5c23f1a89d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:59.738055 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ff447-6fdc-423b-823c-4e5c23f1a89d admin admin] PATH_INFO: `/v3/projects/ea577858f8ad425c958f1cdbecbdc934` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:59.739463 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ff447-6fdc-423b-823c-4e5c23f1a89d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea577858f8ad425c958f1cdbecbdc934)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:30:59.742224 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ff447-6fdc-423b-823c-4e5c23f1a89d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:30:59.824723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ff6ff447-6fdc-423b-823c-4e5c23f1a89d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea577858f8ad425c958f1cdbecbdc934', 'request_id': 'req-ff6ff447-6fdc-423b-823c-4e5c23f1a89d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:59.835660 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ff6ff447-6fdc-423b-823c-4e5c23f1a89d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea577858f8ad425c958f1cdbecbdc934', 'request_id': 'req-ff6ff447-6fdc-423b-823c-4e5c23f1a89d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:59.844439 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ff6ff447-6fdc-423b-823c-4e5c23f1a89d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea577858f8ad425c958f1cdbecbdc934', 'request_id': 'req-ff6ff447-6fdc-423b-823c-4e5c23f1a89d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:59.869399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ff6ff447-6fdc-423b-823c-4e5c23f1a89d admin admin] The token cache is being invalidate because project ea577858f8ad425c958f1cdbecbdc934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:30:59.869645 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ff6ff447-6fdc-423b-823c-4e5c23f1a89d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:59.871432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-ff6ff447-6fdc-423b-823c-4e5c23f1a89d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:30:59.876457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1360/2735] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:30:59 2022] DELETE /identity/v3/projects/ea577858f8ad425c958f1cdbecbdc934 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:30:59.879479 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27346e-1c05-48d2-8117-ebae158805aa None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:59.879993 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27346e-1c05-48d2-8117-ebae158805aa None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:59.880111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27346e-1c05-48d2-8117-ebae158805aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:30:59.912099 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-9a27346e-1c05-48d2-8117-ebae158805aa None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:30:59.913644 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-9a27346e-1c05-48d2-8117-ebae158805aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:30:59.990207 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1376/2736] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:30:59.999280 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436a64a-571f-4050-9951-135926cb04f8 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:30:59.999565 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436a64a-571f-4050-9951-135926cb04f8 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:30:59.999891 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436a64a-571f-4050-9951-135926cb04f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:00.039491 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-3436a64a-571f-4050-9951-135926cb04f8 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:31:00.041037 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3436a64a-571f-4050-9951-135926cb04f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:00.053440 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1361/2737] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:31:00.056462 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27346e-1c05-48d2-8117-ebae158805aa None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:00.064341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:31:00.070184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27346e-1c05-48d2-8117-ebae158805aa None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:00.095932 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec3884-a3e6-4fc9-b976-cbc6dcd88a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:00.096819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec3884-a3e6-4fc9-b976-cbc6dcd88a9d admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:00.097233 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec3884-a3e6-4fc9-b976-cbc6dcd88a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:00.097677 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec3884-a3e6-4fc9-b976-cbc6dcd88a9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:00.098606 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec3884-a3e6-4fc9-b976-cbc6dcd88a9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:31:00.102831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec3884-a3e6-4fc9-b976-cbc6dcd88a9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:00.104000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec3884-a3e6-4fc9-b976-cbc6dcd88a9d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:00.121644 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1377/2738] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 11:31:00.125194 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436a64a-571f-4050-9951-135926cb04f8 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:00.136543 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436a64a-571f-4050-9951-135926cb04f8 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:00.144257 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc669bcd-3284-434b-a516-ca705a6d60f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:00.144873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc669bcd-3284-434b-a516-ca705a6d60f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:00.145190 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc669bcd-3284-434b-a516-ca705a6d60f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:00.145455 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc669bcd-3284-434b-a516-ca705a6d60f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:00.146094 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc669bcd-3284-434b-a516-ca705a6d60f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:00.152067 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc669bcd-3284-434b-a516-ca705a6d60f0 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:00.207304 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1362/2739] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:31:00 2022] POST /identity/v3/projects => generated 367 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:31:00.209201 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec3884-a3e6-4fc9-b976-cbc6dcd88a9d admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:00.221625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec3884-a3e6-4fc9-b976-cbc6dcd88a9d admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:00.282879 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc669bcd-3284-434b-a516-ca705a6d60f0 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:00.285687 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-fc669bcd-3284-434b-a516-ca705a6d60f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:00.309153 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d2715-53a4-4848-8075-af0c3bc3e0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:00.309766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d2715-53a4-4848-8075-af0c3bc3e0bb admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:00.310047 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d2715-53a4-4848-8075-af0c3bc3e0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:00.310316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d2715-53a4-4848-8075-af0c3bc3e0bb admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:00.311124 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d2715-53a4-4848-8075-af0c3bc3e0bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:00.313277 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d2715-53a4-4848-8075-af0c3bc3e0bb admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:00.346920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1378/2740] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:31:00 2022] POST /identity/v3/users => generated 284 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:31:00.350874 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d2715-53a4-4848-8075-af0c3bc3e0bb admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:00.362904 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d2715-53a4-4848-8075-af0c3bc3e0bb admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:00.363697 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc669bcd-3284-434b-a516-ca705a6d60f0 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:00.367989 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e3cb9-f8ef-4843-a310-82f9e2c47734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:00.368590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f29f1f-b9e2-4613-a6f9-91ebea343a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:00.369068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e3cb9-f8ef-4843-a310-82f9e2c47734 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:00.369424 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e3cb9-f8ef-4843-a310-82f9e2c47734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:00.369540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f29f1f-b9e2-4613-a6f9-91ebea343a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:00.369827 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e3cb9-f8ef-4843-a310-82f9e2c47734 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:00.369930 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f29f1f-b9e2-4613-a6f9-91ebea343a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:00.370348 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f29f1f-b9e2-4613-a6f9-91ebea343a39 admin admin] PATH_INFO: `/v3/users/6e8793d50fce45f08c0c4000b7ae75e4` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:00.370675 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e3cb9-f8ef-4843-a310-82f9e2c47734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:00.371739 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f29f1f-b9e2-4613-a6f9-91ebea343a39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e8793d50fce45f08c0c4000b7ae75e4)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:00.372511 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e3cb9-f8ef-4843-a310-82f9e2c47734 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:00.373852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f29f1f-b9e2-4613-a6f9-91ebea343a39 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:00.381747 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:31:00.390236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1379/2741] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:31:00.392423 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e3cb9-f8ef-4843-a310-82f9e2c47734 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:00.408718 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e3cb9-f8ef-4843-a310-82f9e2c47734 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:00.415218 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cad84-90fa-488d-bb9a-df68686e6dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:00.415979 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cad84-90fa-488d-bb9a-df68686e6dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:00.416454 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cad84-90fa-488d-bb9a-df68686e6dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:00.416769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cad84-90fa-488d-bb9a-df68686e6dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:00.417690 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cad84-90fa-488d-bb9a-df68686e6dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:00.419467 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cad84-90fa-488d-bb9a-df68686e6dd7 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:00.431208 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1380/2742] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:31:00.434683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-19f29f1f-b9e2-4613-a6f9-91ebea343a39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e8793d50fce45f08c0c4000b7ae75e4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:00.437005 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cad84-90fa-488d-bb9a-df68686e6dd7 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:00.444047 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-19f29f1f-b9e2-4613-a6f9-91ebea343a39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e8793d50fce45f08c0c4000b7ae75e4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:00.444798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-19f29f1f-b9e2-4613-a6f9-91ebea343a39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e8793d50fce45f08c0c4000b7ae75e4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:00.445494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-19f29f1f-b9e2-4613-a6f9-91ebea343a39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e8793d50fce45f08c0c4000b7ae75e4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:00.450952 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cad84-90fa-488d-bb9a-df68686e6dd7 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:00.456336 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b93d0e-573b-4a1b-b085-d2d86af47700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:00.457668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b93d0e-573b-4a1b-b085-d2d86af47700 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:00.458428 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b93d0e-573b-4a1b-b085-d2d86af47700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:00.458717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b93d0e-573b-4a1b-b085-d2d86af47700 admin admin] PATH_INFO: `/v3/projects/07c0dd5148c74b32abdc5bce068b1c71/users/2ddf2d8e04ea4982b048183ef99bf107/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:00.460827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-19f29f1f-b9e2-4613-a6f9-91ebea343a39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e8793d50fce45f08c0c4000b7ae75e4'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:00.476590 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b93d0e-573b-4a1b-b085-d2d86af47700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07c0dd5148c74b32abdc5bce068b1c71, user_id=2ddf2d8e04ea4982b048183ef99bf107, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:00.481200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1363/2743] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:31:00 2022] DELETE /identity/v3/users/6e8793d50fce45f08c0c4000b7ae75e4 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:31:00.481373 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b93d0e-573b-4a1b-b085-d2d86af47700 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:00.495121 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1381/2744] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:31:00 2022] PUT /identity/v3/projects/07c0dd5148c74b32abdc5bce068b1c71/users/2ddf2d8e04ea4982b048183ef99bf107/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:31:00.505551 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdaaf0-f985-4b85-8305-262d86900247 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:00.505844 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdaaf0-f985-4b85-8305-262d86900247 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:00.506121 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdaaf0-f985-4b85-8305-262d86900247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:00.545299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-69cdaaf0-f985-4b85-8305-262d86900247 None None] MFA Rules not processed for user `2ddf2d8e04ea4982b048183ef99bf107`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:31:00.546686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-69cdaaf0-f985-4b85-8305-262d86900247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:00.623421 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1364/2745] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:31:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:31:00.635246 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b93d0e-573b-4a1b-b085-d2d86af47700 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:00.637632 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-86b93d0e-573b-4a1b-b085-d2d86af47700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:00.672999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdaaf0-f985-4b85-8305-262d86900247 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:00.674270 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-69cdaaf0-f985-4b85-8305-262d86900247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:00.748587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b93d0e-573b-4a1b-b085-d2d86af47700 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:00.753977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db3825-1f7e-4f30-8860-597d68d55dfc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88670b0e1a0f460d8f74917c92db9b9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:00.754589 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db3825-1f7e-4f30-8860-597d68d55dfc service glance] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:00.754868 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db3825-1f7e-4f30-8860-597d68d55dfc service glance] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:00.755141 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db3825-1f7e-4f30-8860-597d68d55dfc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:00.766519 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db3825-1f7e-4f30-8860-597d68d55dfc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:00.768424 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db3825-1f7e-4f30-8860-597d68d55dfc service glance] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:00.770411 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdaaf0-f985-4b85-8305-262d86900247 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:00.776200 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2aa6ac-b5db-49e6-b349-1a4c904f0863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:00.778084 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2aa6ac-b5db-49e6-b349-1a4c904f0863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:00.778373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2aa6ac-b5db-49e6-b349-1a4c904f0863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:00.778650 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2aa6ac-b5db-49e6-b349-1a4c904f0863 admin admin] PATH_INFO: `/v3/projects/221fd300d74c4248a1bfc9b785476477` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:00.779913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2aa6ac-b5db-49e6-b349-1a4c904f0863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=221fd300d74c4248a1bfc9b785476477)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:00.782259 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2aa6ac-b5db-49e6-b349-1a4c904f0863 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:00.788060 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1382/2746] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:31:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:31:00.807367 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:31:00.847210 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4a2aa6ac-b5db-49e6-b349-1a4c904f0863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '221fd300d74c4248a1bfc9b785476477', 'request_id': 'req-4a2aa6ac-b5db-49e6-b349-1a4c904f0863'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:00.858566 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4a2aa6ac-b5db-49e6-b349-1a4c904f0863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '221fd300d74c4248a1bfc9b785476477', 'request_id': 'req-4a2aa6ac-b5db-49e6-b349-1a4c904f0863'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:00.866735 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4a2aa6ac-b5db-49e6-b349-1a4c904f0863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '221fd300d74c4248a1bfc9b785476477', 'request_id': 'req-4a2aa6ac-b5db-49e6-b349-1a4c904f0863'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:00.882685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4a2aa6ac-b5db-49e6-b349-1a4c904f0863 admin admin] The token cache is being invalidate because project 221fd300d74c4248a1bfc9b785476477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:31:00.882990 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4a2aa6ac-b5db-49e6-b349-1a4c904f0863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:00.883682 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4a2aa6ac-b5db-49e6-b349-1a4c904f0863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:00.885506 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1365/2747] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:31:00 2022] DELETE /identity/v3/projects/221fd300d74c4248a1bfc9b785476477 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:31:00.889380 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db3825-1f7e-4f30-8860-597d68d55dfc service glance] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:00.900495 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db3825-1f7e-4f30-8860-597d68d55dfc service glance] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:00.940550 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:31:00.982981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e15ecb-6514-45bc-9994-0fb7a941f0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:00.983629 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e15ecb-6514-45bc-9994-0fb7a941f0ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:00.983940 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e15ecb-6514-45bc-9994-0fb7a941f0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:00.984206 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e15ecb-6514-45bc-9994-0fb7a941f0ad admin admin] PATH_INFO: `/v3/users/c0d9f7bb5d334a3ab57f1ec0e1bc6276` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:00.985554 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e15ecb-6514-45bc-9994-0fb7a941f0ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0d9f7bb5d334a3ab57f1ec0e1bc6276)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:00.987465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e15ecb-6514-45bc-9994-0fb7a941f0ad admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:01.044386 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-37e15ecb-6514-45bc-9994-0fb7a941f0ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0d9f7bb5d334a3ab57f1ec0e1bc6276'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:01.045924 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-37e15ecb-6514-45bc-9994-0fb7a941f0ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0d9f7bb5d334a3ab57f1ec0e1bc6276'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:01.045924 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-37e15ecb-6514-45bc-9994-0fb7a941f0ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0d9f7bb5d334a3ab57f1ec0e1bc6276'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:01.053431 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-37e15ecb-6514-45bc-9994-0fb7a941f0ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0d9f7bb5d334a3ab57f1ec0e1bc6276'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:01.067422 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-37e15ecb-6514-45bc-9994-0fb7a941f0ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0d9f7bb5d334a3ab57f1ec0e1bc6276'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:01.086388 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1383/2748] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:31:00 2022] DELETE /identity/v3/users/c0d9f7bb5d334a3ab57f1ec0e1bc6276 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:31:01.296024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2aa6ac-b5db-49e6-b349-1a4c904f0863 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:01.297457 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4a2aa6ac-b5db-49e6-b349-1a4c904f0863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:01.315614 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2aa6ac-b5db-49e6-b349-1a4c904f0863 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:01.387650 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9554f3-d8ed-44e6-95f3-48575358b5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:01.388314 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9554f3-d8ed-44e6-95f3-48575358b5e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:01.388595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9554f3-d8ed-44e6-95f3-48575358b5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:01.388867 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9554f3-d8ed-44e6-95f3-48575358b5e3 admin admin] PATH_INFO: `/v3/projects/8b760388f536433ba529aa20b48b8c63` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:01.389980 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9554f3-d8ed-44e6-95f3-48575358b5e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b760388f536433ba529aa20b48b8c63)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:01.391908 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9554f3-d8ed-44e6-95f3-48575358b5e3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:01.443784 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4c9554f3-d8ed-44e6-95f3-48575358b5e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b760388f536433ba529aa20b48b8c63', 'request_id': 'req-4c9554f3-d8ed-44e6-95f3-48575358b5e3'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:01.452983 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4c9554f3-d8ed-44e6-95f3-48575358b5e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b760388f536433ba529aa20b48b8c63', 'request_id': 'req-4c9554f3-d8ed-44e6-95f3-48575358b5e3'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:01.461723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4c9554f3-d8ed-44e6-95f3-48575358b5e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b760388f536433ba529aa20b48b8c63', 'request_id': 'req-4c9554f3-d8ed-44e6-95f3-48575358b5e3'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:01.477316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4c9554f3-d8ed-44e6-95f3-48575358b5e3 admin admin] The token cache is being invalidate because project 8b760388f536433ba529aa20b48b8c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:31:01.477726 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4c9554f3-d8ed-44e6-95f3-48575358b5e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:01.478646 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4c9554f3-d8ed-44e6-95f3-48575358b5e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:01.480207 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1366/2749] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:31:01 2022] DELETE /identity/v3/projects/8b760388f536433ba529aa20b48b8c63 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:31:11.079668 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:31:11.474327 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=68,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:31:32.161961 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e15ecb-6514-45bc-9994-0fb7a941f0ad admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:32.163407 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-37e15ecb-6514-45bc-9994-0fb7a941f0ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:32.247132 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e15ecb-6514-45bc-9994-0fb7a941f0ad admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:32.301505 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f58266-3bc2-499f-b478-9f383294db24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:32.302118 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f58266-3bc2-499f-b478-9f383294db24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:32.302391 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f58266-3bc2-499f-b478-9f383294db24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:32.302659 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f58266-3bc2-499f-b478-9f383294db24 admin admin] PATH_INFO: `/v3/users/2ddf2d8e04ea4982b048183ef99bf107` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:32.303785 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f58266-3bc2-499f-b478-9f383294db24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ddf2d8e04ea4982b048183ef99bf107)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:32.305684 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f58266-3bc2-499f-b478-9f383294db24 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:32.350279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-68f58266-3bc2-499f-b478-9f383294db24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ddf2d8e04ea4982b048183ef99bf107'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:32.350927 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-68f58266-3bc2-499f-b478-9f383294db24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ddf2d8e04ea4982b048183ef99bf107'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:32.351536 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-68f58266-3bc2-499f-b478-9f383294db24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ddf2d8e04ea4982b048183ef99bf107'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:32.359080 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-68f58266-3bc2-499f-b478-9f383294db24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ddf2d8e04ea4982b048183ef99bf107'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:32.370720 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-68f58266-3bc2-499f-b478-9f383294db24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ddf2d8e04ea4982b048183ef99bf107'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:32.383470 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1384/2750] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:31:32 2022] DELETE /identity/v3/users/2ddf2d8e04ea4982b048183ef99bf107 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:31:32.387992 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc167b-d95e-42eb-92ee-900b40ef40da None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:32.388520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc167b-d95e-42eb-92ee-900b40ef40da None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:32.388520 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc167b-d95e-42eb-92ee-900b40ef40da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:32.416198 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-26bc167b-d95e-42eb-92ee-900b40ef40da None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:31:32.417330 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-26bc167b-d95e-42eb-92ee-900b40ef40da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:32.474769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1367/2751] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:31:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:31:32.484971 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f58266-3bc2-499f-b478-9f383294db24 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:32.486374 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-68f58266-3bc2-499f-b478-9f383294db24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:32.506645 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:31:32.560468 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f58266-3bc2-499f-b478-9f383294db24 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:32.567267 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2c311-f882-4c90-93ac-1831d7a336b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:32.568507 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2c311-f882-4c90-93ac-1831d7a336b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:32.568799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2c311-f882-4c90-93ac-1831d7a336b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:32.569074 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2c311-f882-4c90-93ac-1831d7a336b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:32.581147 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2c311-f882-4c90-93ac-1831d7a336b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:32.582564 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2c311-f882-4c90-93ac-1831d7a336b7 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:32.592486 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1385/2752] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:31:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:31:32.861474 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc167b-d95e-42eb-92ee-900b40ef40da None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:32.862885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-26bc167b-d95e-42eb-92ee-900b40ef40da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:32.874827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc167b-d95e-42eb-92ee-900b40ef40da None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:32.879552 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4815ab-ffb1-4c5f-968d-d7e161ba882a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:32.880196 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4815ab-ffb1-4c5f-968d-d7e161ba882a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:32.880483 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4815ab-ffb1-4c5f-968d-d7e161ba882a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:32.880737 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4815ab-ffb1-4c5f-968d-d7e161ba882a admin admin] PATH_INFO: `/v3/projects/07c0dd5148c74b32abdc5bce068b1c71` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:32.882744 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4815ab-ffb1-4c5f-968d-d7e161ba882a admin admin] RBAC: Authorizing `identity:delete_project(project_id=07c0dd5148c74b32abdc5bce068b1c71)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:32.884680 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4815ab-ffb1-4c5f-968d-d7e161ba882a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:32.933576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8d4815ab-ffb1-4c5f-968d-d7e161ba882a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07c0dd5148c74b32abdc5bce068b1c71', 'request_id': 'req-8d4815ab-ffb1-4c5f-968d-d7e161ba882a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:32.940173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8d4815ab-ffb1-4c5f-968d-d7e161ba882a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07c0dd5148c74b32abdc5bce068b1c71', 'request_id': 'req-8d4815ab-ffb1-4c5f-968d-d7e161ba882a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:32.945942 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8d4815ab-ffb1-4c5f-968d-d7e161ba882a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07c0dd5148c74b32abdc5bce068b1c71', 'request_id': 'req-8d4815ab-ffb1-4c5f-968d-d7e161ba882a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:32.958957 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8d4815ab-ffb1-4c5f-968d-d7e161ba882a admin admin] The token cache is being invalidate because project 07c0dd5148c74b32abdc5bce068b1c71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:31:32.959248 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8d4815ab-ffb1-4c5f-968d-d7e161ba882a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:32.959895 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-8d4815ab-ffb1-4c5f-968d-d7e161ba882a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:32.961469 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1368/2753] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:31:32 2022] DELETE /identity/v3/projects/07c0dd5148c74b32abdc5bce068b1c71 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:31:32.966307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729b9e7-2184-4055-a055-3fcb9b43a700 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:32.966594 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729b9e7-2184-4055-a055-3fcb9b43a700 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:32.966866 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729b9e7-2184-4055-a055-3fcb9b43a700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:33.014526 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-2729b9e7-2184-4055-a055-3fcb9b43a700 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:31:33.015713 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2729b9e7-2184-4055-a055-3fcb9b43a700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:33.072508 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1386/2754] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:31:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:31:33.136231 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3302656-b301-4821-97f6-60c275849c97 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:33.136518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3302656-b301-4821-97f6-60c275849c97 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:33.136798 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3302656-b301-4821-97f6-60c275849c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:33.157191 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-d3302656-b301-4821-97f6-60c275849c97 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:31:33.158405 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-d3302656-b301-4821-97f6-60c275849c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:33.166322 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1369/2755] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:31:33.170515 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729b9e7-2184-4055-a055-3fcb9b43a700 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:33.180152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729b9e7-2184-4055-a055-3fcb9b43a700 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:33.185297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5fd642-cac2-408d-81ff-396e2647a8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:33.185824 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fd642-cac2-408d-81ff-396e2647a8cc admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:33.186102 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fd642-cac2-408d-81ff-396e2647a8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:33.186369 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fd642-cac2-408d-81ff-396e2647a8cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:33.187000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fd642-cac2-408d-81ff-396e2647a8cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81461) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:31:33.187299 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fd642-cac2-408d-81ff-396e2647a8cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:33.188910 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fd642-cac2-408d-81ff-396e2647a8cc admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:33.201003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1387/2756] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:31:33.205115 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3302656-b301-4821-97f6-60c275849c97 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:33.214165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3302656-b301-4821-97f6-60c275849c97 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:33.218818 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea9359-cce3-4466-aabc-1393e148ded9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:33.219401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea9359-cce3-4466-aabc-1393e148ded9 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:33.219680 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea9359-cce3-4466-aabc-1393e148ded9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:33.219992 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea9359-cce3-4466-aabc-1393e148ded9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:33.220619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea9359-cce3-4466-aabc-1393e148ded9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:33.222509 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea9359-cce3-4466-aabc-1393e148ded9 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:33.248826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1370/2757] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:31:33 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:31:33.252784 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5fd642-cac2-408d-81ff-396e2647a8cc admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:33.263314 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5fd642-cac2-408d-81ff-396e2647a8cc admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:33.317215 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078aa3a-92d2-4a1c-8656-74e68dd79617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:33.317798 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078aa3a-92d2-4a1c-8656-74e68dd79617 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:33.318078 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078aa3a-92d2-4a1c-8656-74e68dd79617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:33.318356 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078aa3a-92d2-4a1c-8656-74e68dd79617 admin admin] PATH_INFO: `/v3/users` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:33.318979 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078aa3a-92d2-4a1c-8656-74e68dd79617 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:33.320923 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078aa3a-92d2-4a1c-8656-74e68dd79617 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:33.337647 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:31:33.344282 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1388/2758] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:31:33 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:31:33.348183 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea9359-cce3-4466-aabc-1393e148ded9 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:33.358846 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea9359-cce3-4466-aabc-1393e148ded9 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:33.363445 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3fbb8-545d-4584-9565-46ba688fee30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:33.364075 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3fbb8-545d-4584-9565-46ba688fee30 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:33.364357 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3fbb8-545d-4584-9565-46ba688fee30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:33.364630 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3fbb8-545d-4584-9565-46ba688fee30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:33.365267 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3fbb8-545d-4584-9565-46ba688fee30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:33.366872 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3fbb8-545d-4584-9565-46ba688fee30 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:33.377624 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1371/2759] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:31:33.381485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078aa3a-92d2-4a1c-8656-74e68dd79617 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:33.391398 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078aa3a-92d2-4a1c-8656-74e68dd79617 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:33.396255 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af57d86-412f-4979-899b-eb328093e3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:33.396945 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af57d86-412f-4979-899b-eb328093e3af admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:33.397314 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af57d86-412f-4979-899b-eb328093e3af admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:33.397693 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af57d86-412f-4979-899b-eb328093e3af admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:33.398600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af57d86-412f-4979-899b-eb328093e3af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:33.400215 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af57d86-412f-4979-899b-eb328093e3af admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:33.410616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1389/2760] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:31:33.414873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3fbb8-545d-4584-9565-46ba688fee30 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:33.425436 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3fbb8-545d-4584-9565-46ba688fee30 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:33.430033 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f3f36-0a92-490e-bb9d-8616a4e0249b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:33.430669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f3f36-0a92-490e-bb9d-8616a4e0249b admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:33.430949 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f3f36-0a92-490e-bb9d-8616a4e0249b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:33.431223 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f3f36-0a92-490e-bb9d-8616a4e0249b admin admin] PATH_INFO: `/v3/projects/e1e88f3a8af54a60807b72beede628fe/users/512412cc094741b4818f8f8c6b6136c9/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:33.446572 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f3f36-0a92-490e-bb9d-8616a4e0249b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1e88f3a8af54a60807b72beede628fe, user_id=512412cc094741b4818f8f8c6b6136c9, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:33.449508 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f3f36-0a92-490e-bb9d-8616a4e0249b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:33.461931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1372/2761] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:31:33 2022] PUT /identity/v3/projects/e1e88f3a8af54a60807b72beede628fe/users/512412cc094741b4818f8f8c6b6136c9/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:31:33.471965 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfea6ec-75d0-4b57-9046-309f5a04184d None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:33.472269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfea6ec-75d0-4b57-9046-309f5a04184d None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:33.472541 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfea6ec-75d0-4b57-9046-309f5a04184d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:33.501034 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-ebfea6ec-75d0-4b57-9046-309f5a04184d None None] MFA Rules not processed for user `512412cc094741b4818f8f8c6b6136c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:31:33.502264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-ebfea6ec-75d0-4b57-9046-309f5a04184d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:33.565791 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1390/2762] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:31:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:31:33.577326 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f3f36-0a92-490e-bb9d-8616a4e0249b admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:33.579006 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-781f3f36-0a92-490e-bb9d-8616a4e0249b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:33.593367 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:31:33.637080 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f3f36-0a92-490e-bb9d-8616a4e0249b admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:33.640842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a31e93-d4a4-4b33-8c98-9ba0b0177da5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:33.641440 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a31e93-d4a4-4b33-8c98-9ba0b0177da5 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:33.641719 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a31e93-d4a4-4b33-8c98-9ba0b0177da5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:33.641996 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a31e93-d4a4-4b33-8c98-9ba0b0177da5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:33.650163 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a31e93-d4a4-4b33-8c98-9ba0b0177da5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:33.651596 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a31e93-d4a4-4b33-8c98-9ba0b0177da5 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:33.660759 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1373/2763] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:31:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:31:36.206427 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfea6ec-75d0-4b57-9046-309f5a04184d None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:36.268254 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfea6ec-75d0-4b57-9046-309f5a04184d None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:36.272778 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462b8cc-9f7c-40c0-a87c-dd90d0921d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:36.273404 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462b8cc-9f7c-40c0-a87c-dd90d0921d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:36.273685 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462b8cc-9f7c-40c0-a87c-dd90d0921d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:36.273958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462b8cc-9f7c-40c0-a87c-dd90d0921d9b admin admin] PATH_INFO: `/v3/users/512412cc094741b4818f8f8c6b6136c9` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:36.275113 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462b8cc-9f7c-40c0-a87c-dd90d0921d9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=512412cc094741b4818f8f8c6b6136c9)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:36.277062 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462b8cc-9f7c-40c0-a87c-dd90d0921d9b admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:36.326404 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-2462b8cc-9f7c-40c0-a87c-dd90d0921d9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '512412cc094741b4818f8f8c6b6136c9'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:36.327065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-2462b8cc-9f7c-40c0-a87c-dd90d0921d9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '512412cc094741b4818f8f8c6b6136c9'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:36.327676 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-2462b8cc-9f7c-40c0-a87c-dd90d0921d9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '512412cc094741b4818f8f8c6b6136c9'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:36.334722 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-2462b8cc-9f7c-40c0-a87c-dd90d0921d9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '512412cc094741b4818f8f8c6b6136c9'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:36.347242 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-2462b8cc-9f7c-40c0-a87c-dd90d0921d9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '512412cc094741b4818f8f8c6b6136c9'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:36.351318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:31:36.373393 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1391/2764] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:31:36 2022] DELETE /identity/v3/users/512412cc094741b4818f8f8c6b6136c9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:31:36.377957 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977060a-2ab6-4847-8b2b-ca5f2e0039d9 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:36.378500 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977060a-2ab6-4847-8b2b-ca5f2e0039d9 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:36.378500 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977060a-2ab6-4847-8b2b-ca5f2e0039d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:36.398495 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9977060a-2ab6-4847-8b2b-ca5f2e0039d9 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:31:36.399723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9977060a-2ab6-4847-8b2b-ca5f2e0039d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:36.459881 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1374/2765] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:31:36.470115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462b8cc-9f7c-40c0-a87c-dd90d0921d9b admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:36.471532 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2462b8cc-9f7c-40c0-a87c-dd90d0921d9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:36.529201 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462b8cc-9f7c-40c0-a87c-dd90d0921d9b admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:36.532878 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10eaf8-5724-4bc9-af9b-6e0b1e177520 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:36.533472 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10eaf8-5724-4bc9-af9b-6e0b1e177520 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:36.533752 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10eaf8-5724-4bc9-af9b-6e0b1e177520 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:36.534025 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10eaf8-5724-4bc9-af9b-6e0b1e177520 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:36.542870 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10eaf8-5724-4bc9-af9b-6e0b1e177520 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:36.544886 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10eaf8-5724-4bc9-af9b-6e0b1e177520 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:36.554273 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1392/2766] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:31:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:31:36.819165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977060a-2ab6-4847-8b2b-ca5f2e0039d9 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:36.821085 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9977060a-2ab6-4847-8b2b-ca5f2e0039d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:36.836212 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977060a-2ab6-4847-8b2b-ca5f2e0039d9 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:36.840889 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393815ec-ba8a-4699-b7ae-369a90f219c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:36.841518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393815ec-ba8a-4699-b7ae-369a90f219c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:36.841794 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393815ec-ba8a-4699-b7ae-369a90f219c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:36.842065 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393815ec-ba8a-4699-b7ae-369a90f219c6 admin admin] PATH_INFO: `/v3/projects/e1e88f3a8af54a60807b72beede628fe` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:36.843248 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393815ec-ba8a-4699-b7ae-369a90f219c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1e88f3a8af54a60807b72beede628fe)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:36.845184 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393815ec-ba8a-4699-b7ae-369a90f219c6 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:36.892424 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-393815ec-ba8a-4699-b7ae-369a90f219c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1e88f3a8af54a60807b72beede628fe', 'request_id': 'req-393815ec-ba8a-4699-b7ae-369a90f219c6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:36.898580 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-393815ec-ba8a-4699-b7ae-369a90f219c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1e88f3a8af54a60807b72beede628fe', 'request_id': 'req-393815ec-ba8a-4699-b7ae-369a90f219c6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:36.904098 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-393815ec-ba8a-4699-b7ae-369a90f219c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1e88f3a8af54a60807b72beede628fe', 'request_id': 'req-393815ec-ba8a-4699-b7ae-369a90f219c6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:36.915584 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-393815ec-ba8a-4699-b7ae-369a90f219c6 admin admin] The token cache is being invalidate because project e1e88f3a8af54a60807b72beede628fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:31:36.915912 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-393815ec-ba8a-4699-b7ae-369a90f219c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:36.916535 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-393815ec-ba8a-4699-b7ae-369a90f219c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:31:36.917983 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1375/2767] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:31:36 2022] DELETE /identity/v3/projects/e1e88f3a8af54a60807b72beede628fe => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:31:41.389065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10eaf8-5724-4bc9-af9b-6e0b1e177520 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:41.390446 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-4e10eaf8-5724-4bc9-af9b-6e0b1e177520 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:41.483557 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10eaf8-5724-4bc9-af9b-6e0b1e177520 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:41.546614 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30941189-495d-4e8f-92e9-2b70c6c8f7aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e7a18dc4a4d4d35a05f1aafbaaf9c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:41.547217 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30941189-495d-4e8f-92e9-2b70c6c8f7aa service placement] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:41.547498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30941189-495d-4e8f-92e9-2b70c6c8f7aa service placement] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:41.547923 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30941189-495d-4e8f-92e9-2b70c6c8f7aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:41.549596 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-30941189-495d-4e8f-92e9-2b70c6c8f7aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:41.561233 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30941189-495d-4e8f-92e9-2b70c6c8f7aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:41.562818 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30941189-495d-4e8f-92e9-2b70c6c8f7aa service placement] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:41.572976 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1393/2768] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:31:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:31:45.103644 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e716e1-0f42-4183-9764-cfe59633c34e None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:45.104220 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e716e1-0f42-4183-9764-cfe59633c34e None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:45.104220 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e716e1-0f42-4183-9764-cfe59633c34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:45.123524 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-20e716e1-0f42-4183-9764-cfe59633c34e None None] MFA Rules not processed for user `7f546dc2d8b24eeea8c6eac968c56806`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:31:45.124775 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-20e716e1-0f42-4183-9764-cfe59633c34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:45.161102 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:31:45.184507 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1376/2769] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:31:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:31:46.937216 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30941189-495d-4e8f-92e9-2b70c6c8f7aa service placement] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:46.939047 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-30941189-495d-4e8f-92e9-2b70c6c8f7aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:46.952580 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=100 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:31:46.998009 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e716e1-0f42-4183-9764-cfe59633c34e None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:46.999431 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-20e716e1-0f42-4183-9764-cfe59633c34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:47.022958 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30941189-495d-4e8f-92e9-2b70c6c8f7aa service placement] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:47.026688 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24109b5f-e5f8-45f2-be8b-4815e92dc736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:47.027910 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24109b5f-e5f8-45f2-be8b-4815e92dc736 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:47.028199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24109b5f-e5f8-45f2-be8b-4815e92dc736 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:47.028472 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24109b5f-e5f8-45f2-be8b-4815e92dc736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:47.030225 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-24109b5f-e5f8-45f2-be8b-4815e92dc736 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:47.041908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24109b5f-e5f8-45f2-be8b-4815e92dc736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:47.043399 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24109b5f-e5f8-45f2-be8b-4815e92dc736 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:47.052604 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1394/2770] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:31:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:31:47.053469 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24109b5f-e5f8-45f2-be8b-4815e92dc736 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:31:47.054803 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-24109b5f-e5f8-45f2-be8b-4815e92dc736 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:31:47.117963 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24109b5f-e5f8-45f2-be8b-4815e92dc736 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:47.118374 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e716e1-0f42-4183-9764-cfe59633c34e None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:31:47.121755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd20824-162f-4d5d-93e7-ccdbc260a654 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:47.122361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd20824-162f-4d5d-93e7-ccdbc260a654 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:47.122642 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd20824-162f-4d5d-93e7-ccdbc260a654 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:47.122920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd20824-162f-4d5d-93e7-ccdbc260a654 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:47.124087 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646a8e3-8c1e-4250-a8ef-a4c7cf58072a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:31:47.124689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646a8e3-8c1e-4250-a8ef-a4c7cf58072a service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:31:47.124965 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646a8e3-8c1e-4250-a8ef-a4c7cf58072a service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:31:47.125239 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646a8e3-8c1e-4250-a8ef-a4c7cf58072a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:31:47.132208 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd20824-162f-4d5d-93e7-ccdbc260a654 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:47.133402 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646a8e3-8c1e-4250-a8ef-a4c7cf58072a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:31:47.133636 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd20824-162f-4d5d-93e7-ccdbc260a654 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:47.135135 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646a8e3-8c1e-4250-a8ef-a4c7cf58072a service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:31:47.143577 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1395/2771] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:31:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:31:47.155669 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1377/2772] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:31:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:31:57.139151 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=56 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:31:57.142240 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=53 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:32:07.901273 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc11de-1f91-41a0-ba2c-59de12d695db None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:32:07.901838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc11de-1f91-41a0-ba2c-59de12d695db None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:32:07.901955 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc11de-1f91-41a0-ba2c-59de12d695db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:32:07.921382 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-d2bc11de-1f91-41a0-ba2c-59de12d695db None None] MFA Rules not processed for user `c23ac045e3db4798be704a4bf37b3c2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:32:07.922587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-d2bc11de-1f91-41a0-ba2c-59de12d695db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:32:07.976414 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1396/2773] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:32:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:32:17.969681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=25 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:33:14.923907 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646a8e3-8c1e-4250-a8ef-a4c7cf58072a service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:14.925276 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-1646a8e3-8c1e-4250-a8ef-a4c7cf58072a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:14.937500 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646a8e3-8c1e-4250-a8ef-a4c7cf58072a service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:14.941169 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c27f3a-86f0-444b-8f25-b68dc22d341a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:14.941770 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c27f3a-86f0-444b-8f25-b68dc22d341a service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:14.942050 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c27f3a-86f0-444b-8f25-b68dc22d341a service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:14.942324 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c27f3a-86f0-444b-8f25-b68dc22d341a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:14.944027 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-33c27f3a-86f0-444b-8f25-b68dc22d341a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:15.010345 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c27f3a-86f0-444b-8f25-b68dc22d341a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:15.011905 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c27f3a-86f0-444b-8f25-b68dc22d341a service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:15.021679 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1378/2774] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:33:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:33:18.064876 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc11de-1f91-41a0-ba2c-59de12d695db None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:18.068454 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-d2bc11de-1f91-41a0-ba2c-59de12d695db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:18.085166 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc11de-1f91-41a0-ba2c-59de12d695db None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:18.090802 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261da6b-da8b-47d4-a329-6028425fdc35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e7a18dc4a4d4d35a05f1aafbaaf9c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:18.091701 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261da6b-da8b-47d4-a329-6028425fdc35 service placement] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:18.092217 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261da6b-da8b-47d4-a329-6028425fdc35 service placement] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:18.092719 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261da6b-da8b-47d4-a329-6028425fdc35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:18.094978 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5261da6b-da8b-47d4-a329-6028425fdc35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:18.111396 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5261da6b-da8b-47d4-a329-6028425fdc35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:18.113556 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5261da6b-da8b-47d4-a329-6028425fdc35 service placement] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:18.137354 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1397/2775] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:33:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:33:18.139082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c27f3a-86f0-444b-8f25-b68dc22d341a service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:18.140893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-33c27f3a-86f0-444b-8f25-b68dc22d341a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:18.164833 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c27f3a-86f0-444b-8f25-b68dc22d341a service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:18.171485 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7d56c-fcb4-4f38-9600-9f7e927e8421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:18.173605 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7d56c-fcb4-4f38-9600-9f7e927e8421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:18.174211 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7d56c-fcb4-4f38-9600-9f7e927e8421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:18.174794 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7d56c-fcb4-4f38-9600-9f7e927e8421 admin admin] PATH_INFO: `/v3/users/c23ac045e3db4798be704a4bf37b3c2c` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:18.176649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7d56c-fcb4-4f38-9600-9f7e927e8421 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c23ac045e3db4798be704a4bf37b3c2c)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:18.180509 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7d56c-fcb4-4f38-9600-9f7e927e8421 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:18.234073 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a0e7d56c-fcb4-4f38-9600-9f7e927e8421 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c23ac045e3db4798be704a4bf37b3c2c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:18.241230 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a0e7d56c-fcb4-4f38-9600-9f7e927e8421 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c23ac045e3db4798be704a4bf37b3c2c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:18.241838 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a0e7d56c-fcb4-4f38-9600-9f7e927e8421 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c23ac045e3db4798be704a4bf37b3c2c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:18.242472 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a0e7d56c-fcb4-4f38-9600-9f7e927e8421 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c23ac045e3db4798be704a4bf37b3c2c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:18.254852 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-a0e7d56c-fcb4-4f38-9600-9f7e927e8421 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c23ac045e3db4798be704a4bf37b3c2c'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:18.266893 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1379/2776] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:33:18 2022] DELETE /identity/v3/users/c23ac045e3db4798be704a4bf37b3c2c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:33:18.438498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261da6b-da8b-47d4-a329-6028425fdc35 service placement] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:18.454544 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261da6b-da8b-47d4-a329-6028425fdc35 service placement] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:18.516185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02958800-8a34-4ba5-bb0a-29baa2dcad46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:18.516810 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02958800-8a34-4ba5-bb0a-29baa2dcad46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:18.517088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02958800-8a34-4ba5-bb0a-29baa2dcad46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:18.517498 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02958800-8a34-4ba5-bb0a-29baa2dcad46 admin admin] PATH_INFO: `/v3/projects/0e5ff724699a4dafb37bc5d5e240f0b2` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:18.519408 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02958800-8a34-4ba5-bb0a-29baa2dcad46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e5ff724699a4dafb37bc5d5e240f0b2)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:18.522820 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02958800-8a34-4ba5-bb0a-29baa2dcad46 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:18.568338 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-02958800-8a34-4ba5-bb0a-29baa2dcad46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e5ff724699a4dafb37bc5d5e240f0b2', 'request_id': 'req-02958800-8a34-4ba5-bb0a-29baa2dcad46'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:18.574643 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-02958800-8a34-4ba5-bb0a-29baa2dcad46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e5ff724699a4dafb37bc5d5e240f0b2', 'request_id': 'req-02958800-8a34-4ba5-bb0a-29baa2dcad46'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:18.580125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-02958800-8a34-4ba5-bb0a-29baa2dcad46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e5ff724699a4dafb37bc5d5e240f0b2', 'request_id': 'req-02958800-8a34-4ba5-bb0a-29baa2dcad46'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:18.593274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-02958800-8a34-4ba5-bb0a-29baa2dcad46 admin admin] The token cache is being invalidate because project 0e5ff724699a4dafb37bc5d5e240f0b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:33:18.593566 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-02958800-8a34-4ba5-bb0a-29baa2dcad46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:18.594191 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-02958800-8a34-4ba5-bb0a-29baa2dcad46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:18.595648 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1398/2777] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:33:18 2022] DELETE /identity/v3/projects/0e5ff724699a4dafb37bc5d5e240f0b2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:33:18.600494 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807626e7-29e9-4606-8913-9eea7f94514f None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:18.600783 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807626e7-29e9-4606-8913-9eea7f94514f None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:18.601079 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807626e7-29e9-4606-8913-9eea7f94514f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:18.621368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-807626e7-29e9-4606-8913-9eea7f94514f None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:33:18.622575 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-807626e7-29e9-4606-8913-9eea7f94514f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:18.681294 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1380/2778] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:33:18.689119 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6e90e-decb-499b-99fd-03849a841241 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:18.689406 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6e90e-decb-499b-99fd-03849a841241 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:18.689683 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6e90e-decb-499b-99fd-03849a841241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:18.710329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-52d6e90e-decb-499b-99fd-03849a841241 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:33:18.711524 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-52d6e90e-decb-499b-99fd-03849a841241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:18.720006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1399/2779] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:33:18.724078 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807626e7-29e9-4606-8913-9eea7f94514f None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:18.734170 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807626e7-29e9-4606-8913-9eea7f94514f None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:18.738711 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc98e1e-8140-4825-bf7c-c05733bc09b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:18.739296 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc98e1e-8140-4825-bf7c-c05733bc09b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:18.739576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc98e1e-8140-4825-bf7c-c05733bc09b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:18.739879 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc98e1e-8140-4825-bf7c-c05733bc09b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:18.740515 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc98e1e-8140-4825-bf7c-c05733bc09b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:33:18.740813 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc98e1e-8140-4825-bf7c-c05733bc09b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:18.742397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc98e1e-8140-4825-bf7c-c05733bc09b2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:18.743908 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:33:18.758351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1381/2780] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:33:18.759680 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6e90e-decb-499b-99fd-03849a841241 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:18.769853 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6e90e-decb-499b-99fd-03849a841241 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:18.774468 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b1f2d-a5fc-432c-a961-6ed5534015b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:18.775050 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b1f2d-a5fc-432c-a961-6ed5534015b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:18.775323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b1f2d-a5fc-432c-a961-6ed5534015b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:18.776126 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b1f2d-a5fc-432c-a961-6ed5534015b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:18.776790 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b1f2d-a5fc-432c-a961-6ed5534015b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:18.778648 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b1f2d-a5fc-432c-a961-6ed5534015b5 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:18.805186 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1400/2781] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:33:18 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:33:18.809621 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc98e1e-8140-4825-bf7c-c05733bc09b2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:18.819605 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc98e1e-8140-4825-bf7c-c05733bc09b2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:18.874718 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed898b6c-109a-4d37-9f3c-e00b2f3b0864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:18.875313 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed898b6c-109a-4d37-9f3c-e00b2f3b0864 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:18.875589 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed898b6c-109a-4d37-9f3c-e00b2f3b0864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:18.875892 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed898b6c-109a-4d37-9f3c-e00b2f3b0864 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:18.876525 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed898b6c-109a-4d37-9f3c-e00b2f3b0864 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:18.878440 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed898b6c-109a-4d37-9f3c-e00b2f3b0864 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:18.899081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1382/2782] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:33:18 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:33:18.903871 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b1f2d-a5fc-432c-a961-6ed5534015b5 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:18.914328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b1f2d-a5fc-432c-a961-6ed5534015b5 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:18.919542 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6c3e0-61f6-433b-ab9c-bb1624df90ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:18.920165 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6c3e0-61f6-433b-ab9c-bb1624df90ed admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:18.920445 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6c3e0-61f6-433b-ab9c-bb1624df90ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:18.920713 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6c3e0-61f6-433b-ab9c-bb1624df90ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:18.921343 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6c3e0-61f6-433b-ab9c-bb1624df90ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:18.922905 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6c3e0-61f6-433b-ab9c-bb1624df90ed admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:18.932988 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1401/2783] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:33:18.936855 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed898b6c-109a-4d37-9f3c-e00b2f3b0864 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:18.947056 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed898b6c-109a-4d37-9f3c-e00b2f3b0864 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:18.952545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275e61b-d0c2-411b-9070-5b0b712fb0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:18.953139 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275e61b-d0c2-411b-9070-5b0b712fb0bd admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:18.953417 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275e61b-d0c2-411b-9070-5b0b712fb0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:18.953689 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275e61b-d0c2-411b-9070-5b0b712fb0bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:18.954328 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275e61b-d0c2-411b-9070-5b0b712fb0bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:18.955998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275e61b-d0c2-411b-9070-5b0b712fb0bd admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:18.967227 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1383/2784] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:33:18.971274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6c3e0-61f6-433b-ab9c-bb1624df90ed admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:18.981274 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6c3e0-61f6-433b-ab9c-bb1624df90ed admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:18.985862 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e85524-1576-45de-a944-977fa263f961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:18.986494 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e85524-1576-45de-a944-977fa263f961 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:18.986769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e85524-1576-45de-a944-977fa263f961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:18.987038 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e85524-1576-45de-a944-977fa263f961 admin admin] PATH_INFO: `/v3/projects/d6b3fad43905435a8c0b13212b50b11b/users/64d1c2e339e045e4a82dcc3c198b6fb3/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:19.001446 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e85524-1576-45de-a944-977fa263f961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6b3fad43905435a8c0b13212b50b11b, user_id=64d1c2e339e045e4a82dcc3c198b6fb3, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:19.004078 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e85524-1576-45de-a944-977fa263f961 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:19.019083 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1402/2785] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:33:18 2022] PUT /identity/v3/projects/d6b3fad43905435a8c0b13212b50b11b/users/64d1c2e339e045e4a82dcc3c198b6fb3/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:33:19.031037 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf6821-bb49-4857-b34b-ea0a24284903 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:19.031340 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf6821-bb49-4857-b34b-ea0a24284903 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:19.031608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf6821-bb49-4857-b34b-ea0a24284903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:19.061406 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9cbf6821-bb49-4857-b34b-ea0a24284903 None None] MFA Rules not processed for user `64d1c2e339e045e4a82dcc3c198b6fb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:33:19.063181 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9cbf6821-bb49-4857-b34b-ea0a24284903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:19.126441 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1384/2786] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:33:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:33:19.130989 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e85524-1576-45de-a944-977fa263f961 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:19.145685 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:33:19.193848 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e85524-1576-45de-a944-977fa263f961 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:19.199288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7cb8b-6493-4e66-9357-7ac2bd535d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:19.199966 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7cb8b-6493-4e66-9357-7ac2bd535d66 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:19.200249 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7cb8b-6493-4e66-9357-7ac2bd535d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:19.200523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7cb8b-6493-4e66-9357-7ac2bd535d66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:19.201143 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7cb8b-6493-4e66-9357-7ac2bd535d66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:19.203032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7cb8b-6493-4e66-9357-7ac2bd535d66 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:19.228906 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1403/2787] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:33:19 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:33:19.232921 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf6821-bb49-4857-b34b-ea0a24284903 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:19.243296 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf6821-bb49-4857-b34b-ea0a24284903 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:19.297148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b3b02-4d65-4961-9371-a8a015794864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:19.297733 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b3b02-4d65-4961-9371-a8a015794864 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:19.298033 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b3b02-4d65-4961-9371-a8a015794864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:19.298299 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b3b02-4d65-4961-9371-a8a015794864 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:19.298910 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b3b02-4d65-4961-9371-a8a015794864 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:19.300862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b3b02-4d65-4961-9371-a8a015794864 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:19.309464 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:33:19.323545 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1385/2788] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:33:19 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:33:19.327503 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7cb8b-6493-4e66-9357-7ac2bd535d66 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:19.337720 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7cb8b-6493-4e66-9357-7ac2bd535d66 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:19.342896 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96cf80b-ff82-4f50-98d0-cc1324a29fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:19.343486 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cf80b-ff82-4f50-98d0-cc1324a29fce admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:19.343782 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cf80b-ff82-4f50-98d0-cc1324a29fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:19.344072 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cf80b-ff82-4f50-98d0-cc1324a29fce admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:19.344706 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96cf80b-ff82-4f50-98d0-cc1324a29fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:19.346294 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96cf80b-ff82-4f50-98d0-cc1324a29fce admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:19.356552 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1404/2789] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:33:19.360656 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b3b02-4d65-4961-9371-a8a015794864 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:19.371041 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b3b02-4d65-4961-9371-a8a015794864 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:19.375553 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b1c93-e02f-4237-9796-30763e080aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:19.376226 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b1c93-e02f-4237-9796-30763e080aac admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:19.376559 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b1c93-e02f-4237-9796-30763e080aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:19.376776 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b1c93-e02f-4237-9796-30763e080aac admin admin] PATH_INFO: `/v3/projects/513cc885525646609e747cf2669d7770/users/f378dba56b714cd7b414be18adbe6c63/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:19.391744 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b1c93-e02f-4237-9796-30763e080aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=513cc885525646609e747cf2669d7770, user_id=f378dba56b714cd7b414be18adbe6c63, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:19.394451 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b1c93-e02f-4237-9796-30763e080aac admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:19.407082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1386/2790] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:33:19 2022] PUT /identity/v3/projects/513cc885525646609e747cf2669d7770/users/f378dba56b714cd7b414be18adbe6c63/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:33:19.411349 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96cf80b-ff82-4f50-98d0-cc1324a29fce admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:19.421335 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96cf80b-ff82-4f50-98d0-cc1324a29fce admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:19.476657 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af119cef-b6ac-4a9e-bc49-87df187e73bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:19.477238 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af119cef-b6ac-4a9e-bc49-87df187e73bb admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:19.477513 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af119cef-b6ac-4a9e-bc49-87df187e73bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:19.477780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af119cef-b6ac-4a9e-bc49-87df187e73bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:19.478401 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af119cef-b6ac-4a9e-bc49-87df187e73bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:19.479992 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af119cef-b6ac-4a9e-bc49-87df187e73bb admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:19.489341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1405/2791] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:33:19.493482 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b1c93-e02f-4237-9796-30763e080aac admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:19.504475 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b1c93-e02f-4237-9796-30763e080aac admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:19.509152 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502c43f-3020-4f5d-9295-bf1836d2d171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:19.510401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502c43f-3020-4f5d-9295-bf1836d2d171 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:19.510682 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502c43f-3020-4f5d-9295-bf1836d2d171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:19.510954 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502c43f-3020-4f5d-9295-bf1836d2d171 admin admin] PATH_INFO: `/v3/projects/513cc885525646609e747cf2669d7770/users/f378dba56b714cd7b414be18adbe6c63/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:19.512922 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502c43f-3020-4f5d-9295-bf1836d2d171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=513cc885525646609e747cf2669d7770, user_id=f378dba56b714cd7b414be18adbe6c63, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:19.515592 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502c43f-3020-4f5d-9295-bf1836d2d171 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:19.525696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1387/2792] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:33:19 2022] PUT /identity/v3/projects/513cc885525646609e747cf2669d7770/users/f378dba56b714cd7b414be18adbe6c63/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:33:19.535711 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcba423-b723-4d3a-97ce-f1a3f355907d None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:19.536018 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcba423-b723-4d3a-97ce-f1a3f355907d None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:19.536293 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcba423-b723-4d3a-97ce-f1a3f355907d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:19.563949 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-7bcba423-b723-4d3a-97ce-f1a3f355907d None None] MFA Rules not processed for user `f378dba56b714cd7b414be18adbe6c63`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:33:19.565151 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-7bcba423-b723-4d3a-97ce-f1a3f355907d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:19.631611 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1406/2793] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:33:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:33:19.643325 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502c43f-3020-4f5d-9295-bf1836d2d171 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:19.645090 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2502c43f-3020-4f5d-9295-bf1836d2d171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:19.722743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502c43f-3020-4f5d-9295-bf1836d2d171 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:19.726401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75959678-a7a8-4fed-9278-652f05d77be7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:19.726998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75959678-a7a8-4fed-9278-652f05d77be7 service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:19.727279 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75959678-a7a8-4fed-9278-652f05d77be7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:19.727557 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75959678-a7a8-4fed-9278-652f05d77be7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:19.737123 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75959678-a7a8-4fed-9278-652f05d77be7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:19.738551 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75959678-a7a8-4fed-9278-652f05d77be7 service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:19.748571 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1388/2794] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:33:19 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:33:19.821900 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcba423-b723-4d3a-97ce-f1a3f355907d None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:19.831843 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcba423-b723-4d3a-97ce-f1a3f355907d None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:19.835454 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7875c-1fda-408b-8fe3-f66fe19fa0f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:19.836080 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7875c-1fda-408b-8fe3-f66fe19fa0f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:19.836366 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7875c-1fda-408b-8fe3-f66fe19fa0f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:19.836638 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7875c-1fda-408b-8fe3-f66fe19fa0f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:19.842065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:33:19.896441 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7875c-1fda-408b-8fe3-f66fe19fa0f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:19.897778 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7875c-1fda-408b-8fe3-f66fe19fa0f5 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:19.906824 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1407/2795] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:33:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:33:22.627505 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75959678-a7a8-4fed-9278-652f05d77be7 service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:22.696713 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75959678-a7a8-4fed-9278-652f05d77be7 service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:22.701762 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f82f18-2fa8-4a54-97bb-0b5c9fcfa6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:22.702388 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f82f18-2fa8-4a54-97bb-0b5c9fcfa6de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:22.702662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f82f18-2fa8-4a54-97bb-0b5c9fcfa6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:22.702933 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f82f18-2fa8-4a54-97bb-0b5c9fcfa6de admin admin] PATH_INFO: `/v3/users/64d1c2e339e045e4a82dcc3c198b6fb3` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:22.704082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f82f18-2fa8-4a54-97bb-0b5c9fcfa6de admin admin] RBAC: Authorizing `identity:delete_user(user_id=64d1c2e339e045e4a82dcc3c198b6fb3)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:22.705992 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f82f18-2fa8-4a54-97bb-0b5c9fcfa6de admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:22.754675 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:33:22.762574 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b7f82f18-2fa8-4a54-97bb-0b5c9fcfa6de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64d1c2e339e045e4a82dcc3c198b6fb3'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:22.769767 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b7f82f18-2fa8-4a54-97bb-0b5c9fcfa6de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64d1c2e339e045e4a82dcc3c198b6fb3'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:22.770409 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b7f82f18-2fa8-4a54-97bb-0b5c9fcfa6de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64d1c2e339e045e4a82dcc3c198b6fb3'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:22.771011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b7f82f18-2fa8-4a54-97bb-0b5c9fcfa6de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64d1c2e339e045e4a82dcc3c198b6fb3'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:22.783656 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-b7f82f18-2fa8-4a54-97bb-0b5c9fcfa6de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64d1c2e339e045e4a82dcc3c198b6fb3'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:22.796349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1389/2796] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:33:22 2022] DELETE /identity/v3/users/64d1c2e339e045e4a82dcc3c198b6fb3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:33:22.800792 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8db56-a58f-403d-864a-aeaa3a257c14 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:22.801271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8db56-a58f-403d-864a-aeaa3a257c14 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:22.801388 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8db56-a58f-403d-864a-aeaa3a257c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:22.821263 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-36b8db56-a58f-403d-864a-aeaa3a257c14 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:33:22.822483 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-36b8db56-a58f-403d-864a-aeaa3a257c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:22.880875 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1408/2797] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:33:22.894560 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f82f18-2fa8-4a54-97bb-0b5c9fcfa6de admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:22.895990 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-b7f82f18-2fa8-4a54-97bb-0b5c9fcfa6de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:22.953633 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f82f18-2fa8-4a54-97bb-0b5c9fcfa6de admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:22.957285 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b21bf5-4b67-4b12-b9e1-9c086a957545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:22.957897 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b21bf5-4b67-4b12-b9e1-9c086a957545 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:22.958163 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b21bf5-4b67-4b12-b9e1-9c086a957545 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:22.958431 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b21bf5-4b67-4b12-b9e1-9c086a957545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:22.967629 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b21bf5-4b67-4b12-b9e1-9c086a957545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:22.969087 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b21bf5-4b67-4b12-b9e1-9c086a957545 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:22.978501 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1390/2798] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:33:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:33:23.253086 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8db56-a58f-403d-864a-aeaa3a257c14 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:23.255305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-36b8db56-a58f-403d-864a-aeaa3a257c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:23.267719 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8db56-a58f-403d-864a-aeaa3a257c14 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:23.272652 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782bc69-922e-4be2-95dd-08976bc01fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:23.273288 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782bc69-922e-4be2-95dd-08976bc01fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:23.273570 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782bc69-922e-4be2-95dd-08976bc01fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:23.273843 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782bc69-922e-4be2-95dd-08976bc01fe8 admin admin] PATH_INFO: `/v3/projects/d6b3fad43905435a8c0b13212b50b11b` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:23.275695 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782bc69-922e-4be2-95dd-08976bc01fe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6b3fad43905435a8c0b13212b50b11b)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:23.278716 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782bc69-922e-4be2-95dd-08976bc01fe8 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:23.334029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-9782bc69-922e-4be2-95dd-08976bc01fe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6b3fad43905435a8c0b13212b50b11b', 'request_id': 'req-9782bc69-922e-4be2-95dd-08976bc01fe8'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:23.341934 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-9782bc69-922e-4be2-95dd-08976bc01fe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6b3fad43905435a8c0b13212b50b11b', 'request_id': 'req-9782bc69-922e-4be2-95dd-08976bc01fe8'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:23.347634 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-9782bc69-922e-4be2-95dd-08976bc01fe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6b3fad43905435a8c0b13212b50b11b', 'request_id': 'req-9782bc69-922e-4be2-95dd-08976bc01fe8'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:23.362485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-9782bc69-922e-4be2-95dd-08976bc01fe8 admin admin] The token cache is being invalidate because project d6b3fad43905435a8c0b13212b50b11b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:33:23.362771 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-9782bc69-922e-4be2-95dd-08976bc01fe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:23.363392 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-9782bc69-922e-4be2-95dd-08976bc01fe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:23.365867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1409/2799] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:33:23 2022] DELETE /identity/v3/projects/d6b3fad43905435a8c0b13212b50b11b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:33:23.370024 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b21bf5-4b67-4b12-b9e1-9c086a957545 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:23.379743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b21bf5-4b67-4b12-b9e1-9c086a957545 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:23.438653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe7ffa-aecd-42cc-bd42-07e0ecddcf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:23.439351 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe7ffa-aecd-42cc-bd42-07e0ecddcf0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:23.439636 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe7ffa-aecd-42cc-bd42-07e0ecddcf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:23.439945 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe7ffa-aecd-42cc-bd42-07e0ecddcf0c admin admin] PATH_INFO: `/v3/users/f378dba56b714cd7b414be18adbe6c63` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:23.441054 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe7ffa-aecd-42cc-bd42-07e0ecddcf0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f378dba56b714cd7b414be18adbe6c63)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:23.443014 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe7ffa-aecd-42cc-bd42-07e0ecddcf0c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:23.488247 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-83fe7ffa-aecd-42cc-bd42-07e0ecddcf0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f378dba56b714cd7b414be18adbe6c63'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:23.495225 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-83fe7ffa-aecd-42cc-bd42-07e0ecddcf0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f378dba56b714cd7b414be18adbe6c63'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:23.495872 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-83fe7ffa-aecd-42cc-bd42-07e0ecddcf0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f378dba56b714cd7b414be18adbe6c63'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:23.496499 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-83fe7ffa-aecd-42cc-bd42-07e0ecddcf0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f378dba56b714cd7b414be18adbe6c63'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:23.501661 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:33:23.509881 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-83fe7ffa-aecd-42cc-bd42-07e0ecddcf0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f378dba56b714cd7b414be18adbe6c63'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:23.523255 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1391/2800] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:33:23 2022] DELETE /identity/v3/users/f378dba56b714cd7b414be18adbe6c63 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:33:23.789342 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782bc69-922e-4be2-95dd-08976bc01fe8 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:23.790844 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-9782bc69-922e-4be2-95dd-08976bc01fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:23.804595 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782bc69-922e-4be2-95dd-08976bc01fe8 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:23.835990 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:33:23.867193 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f213969-1add-4074-8791-e8204c56dbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:23.867849 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f213969-1add-4074-8791-e8204c56dbdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:23.868131 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f213969-1add-4074-8791-e8204c56dbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:23.868403 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f213969-1add-4074-8791-e8204c56dbdf admin admin] PATH_INFO: `/v3/projects/513cc885525646609e747cf2669d7770` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:23.869575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f213969-1add-4074-8791-e8204c56dbdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=513cc885525646609e747cf2669d7770)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:23.871575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f213969-1add-4074-8791-e8204c56dbdf admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:23.916624 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-7f213969-1add-4074-8791-e8204c56dbdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '513cc885525646609e747cf2669d7770', 'request_id': 'req-7f213969-1add-4074-8791-e8204c56dbdf'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:23.923328 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-7f213969-1add-4074-8791-e8204c56dbdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '513cc885525646609e747cf2669d7770', 'request_id': 'req-7f213969-1add-4074-8791-e8204c56dbdf'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:23.928992 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-7f213969-1add-4074-8791-e8204c56dbdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '513cc885525646609e747cf2669d7770', 'request_id': 'req-7f213969-1add-4074-8791-e8204c56dbdf'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:23.940819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-7f213969-1add-4074-8791-e8204c56dbdf admin admin] The token cache is being invalidate because project 513cc885525646609e747cf2669d7770 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:33:23.941123 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-7f213969-1add-4074-8791-e8204c56dbdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:23.941734 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-7f213969-1add-4074-8791-e8204c56dbdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:23.943195 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1410/2801] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:33:23 2022] DELETE /identity/v3/projects/513cc885525646609e747cf2669d7770 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:33:23.948513 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ea930-b492-4b7b-85dd-e5b570ce3d6c None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:23.948997 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ea930-b492-4b7b-85dd-e5b570ce3d6c None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:23.949113 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ea930-b492-4b7b-85dd-e5b570ce3d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:23.968995 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-314ea930-b492-4b7b-85dd-e5b570ce3d6c None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:33:23.970206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-314ea930-b492-4b7b-85dd-e5b570ce3d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:24.030009 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1392/2802] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:33:24.037791 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b06f4b-ee9f-497c-8b41-70b7c15f66aa None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:24.038080 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b06f4b-ee9f-497c-8b41-70b7c15f66aa None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:24.038354 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b06f4b-ee9f-497c-8b41-70b7c15f66aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:24.058799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-61b06f4b-ee9f-497c-8b41-70b7c15f66aa None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:33:24.060035 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-61b06f4b-ee9f-497c-8b41-70b7c15f66aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:24.067990 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1411/2803] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:33:24.072513 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ea930-b492-4b7b-85dd-e5b570ce3d6c None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:24.082164 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ea930-b492-4b7b-85dd-e5b570ce3d6c None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:24.086751 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b64d84-729a-4ee5-aaa5-0a73ac2c503f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:24.087342 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b64d84-729a-4ee5-aaa5-0a73ac2c503f admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:24.087623 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b64d84-729a-4ee5-aaa5-0a73ac2c503f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:24.087953 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b64d84-729a-4ee5-aaa5-0a73ac2c503f admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:24.089093 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b64d84-729a-4ee5-aaa5-0a73ac2c503f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:33:24.089397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b64d84-729a-4ee5-aaa5-0a73ac2c503f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:24.090978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b64d84-729a-4ee5-aaa5-0a73ac2c503f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:24.102666 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1393/2804] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:33:24.107371 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b06f4b-ee9f-497c-8b41-70b7c15f66aa None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:24.116350 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b06f4b-ee9f-497c-8b41-70b7c15f66aa None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:24.120926 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad4f57-330c-4496-9620-73b0ff3678e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:24.121448 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad4f57-330c-4496-9620-73b0ff3678e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:24.121723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad4f57-330c-4496-9620-73b0ff3678e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:24.121988 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad4f57-330c-4496-9620-73b0ff3678e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:24.122602 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad4f57-330c-4496-9620-73b0ff3678e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:24.124506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad4f57-330c-4496-9620-73b0ff3678e6 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:24.160236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1412/2805] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:33:24 2022] POST /identity/v3/projects => generated 393 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:33:24.164348 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b64d84-729a-4ee5-aaa5-0a73ac2c503f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:24.175316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b64d84-729a-4ee5-aaa5-0a73ac2c503f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:24.233230 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d3557-e106-4323-b6f9-a261b3a8cff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:24.233830 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d3557-e106-4323-b6f9-a261b3a8cff1 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:24.234107 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d3557-e106-4323-b6f9-a261b3a8cff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:24.234389 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d3557-e106-4323-b6f9-a261b3a8cff1 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:24.235005 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d3557-e106-4323-b6f9-a261b3a8cff1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:24.237527 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d3557-e106-4323-b6f9-a261b3a8cff1 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:24.257427 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1394/2806] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:33:24 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:33:24.262200 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad4f57-330c-4496-9620-73b0ff3678e6 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:24.272401 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad4f57-330c-4496-9620-73b0ff3678e6 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:24.277764 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c889a343-ad9f-4fde-8fb8-dc2447233893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:24.278349 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889a343-ad9f-4fde-8fb8-dc2447233893 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:24.278622 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889a343-ad9f-4fde-8fb8-dc2447233893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:24.278889 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889a343-ad9f-4fde-8fb8-dc2447233893 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:24.279516 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c889a343-ad9f-4fde-8fb8-dc2447233893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:24.281182 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c889a343-ad9f-4fde-8fb8-dc2447233893 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:24.290930 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1413/2807] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:33:24.294780 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d3557-e106-4323-b6f9-a261b3a8cff1 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:24.304467 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d3557-e106-4323-b6f9-a261b3a8cff1 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:24.309061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b56a10-310f-47de-a439-a79c673c59b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:24.309637 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b56a10-310f-47de-a439-a79c673c59b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:24.309913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b56a10-310f-47de-a439-a79c673c59b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:24.310178 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b56a10-310f-47de-a439-a79c673c59b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:24.310807 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b56a10-310f-47de-a439-a79c673c59b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:24.312403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b56a10-310f-47de-a439-a79c673c59b3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:24.323601 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1395/2808] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:33:24.327702 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c889a343-ad9f-4fde-8fb8-dc2447233893 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:24.337656 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c889a343-ad9f-4fde-8fb8-dc2447233893 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:24.342838 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973c46f-748f-4c8e-b52a-b91eecda5d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:24.343472 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973c46f-748f-4c8e-b52a-b91eecda5d1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:24.343755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973c46f-748f-4c8e-b52a-b91eecda5d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:24.344070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973c46f-748f-4c8e-b52a-b91eecda5d1f admin admin] PATH_INFO: `/v3/projects/05a2fd94b42847628da44d04c8e9f3e6/users/eb172b7c8ff049e1aaff449d2181ba5d/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:24.358588 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7973c46f-748f-4c8e-b52a-b91eecda5d1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=05a2fd94b42847628da44d04c8e9f3e6, user_id=eb172b7c8ff049e1aaff449d2181ba5d, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:24.361255 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7973c46f-748f-4c8e-b52a-b91eecda5d1f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:24.373956 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1414/2809] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:33:24 2022] PUT /identity/v3/projects/05a2fd94b42847628da44d04c8e9f3e6/users/eb172b7c8ff049e1aaff449d2181ba5d/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:33:24.385236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc42905-ddda-488b-902f-21a2df2c1a29 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:24.386022 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc42905-ddda-488b-902f-21a2df2c1a29 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:24.386320 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc42905-ddda-488b-902f-21a2df2c1a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:24.415135 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-6fc42905-ddda-488b-902f-21a2df2c1a29 None None] MFA Rules not processed for user `eb172b7c8ff049e1aaff449d2181ba5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:33:24.416379 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-6fc42905-ddda-488b-902f-21a2df2c1a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:24.448696 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:33:24.482859 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1396/2810] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:33:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:33:24.487497 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973c46f-748f-4c8e-b52a-b91eecda5d1f admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:24.534602 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:33:24.549836 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973c46f-748f-4c8e-b52a-b91eecda5d1f admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:24.554531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b131db-2209-4ee1-8bf4-b09c394fe5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:24.555121 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b131db-2209-4ee1-8bf4-b09c394fe5da admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:24.555405 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b131db-2209-4ee1-8bf4-b09c394fe5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:24.555866 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b131db-2209-4ee1-8bf4-b09c394fe5da admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:24.556331 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b131db-2209-4ee1-8bf4-b09c394fe5da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:24.558797 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b131db-2209-4ee1-8bf4-b09c394fe5da admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:24.584300 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1415/2811] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:33:24 2022] POST /identity/v3/projects => generated 393 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:33:24.588940 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc42905-ddda-488b-902f-21a2df2c1a29 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:24.599346 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc42905-ddda-488b-902f-21a2df2c1a29 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:24.654951 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e4fe1-b2e5-48c7-9f55-f8a5f673981c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:24.655487 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e4fe1-b2e5-48c7-9f55-f8a5f673981c admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:24.655770 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e4fe1-b2e5-48c7-9f55-f8a5f673981c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:24.656069 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e4fe1-b2e5-48c7-9f55-f8a5f673981c admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:24.656712 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e4fe1-b2e5-48c7-9f55-f8a5f673981c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:24.658631 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e4fe1-b2e5-48c7-9f55-f8a5f673981c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:24.680428 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1397/2812] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:33:24 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:33:24.684362 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b131db-2209-4ee1-8bf4-b09c394fe5da admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:24.694482 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b131db-2209-4ee1-8bf4-b09c394fe5da admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:24.699424 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7c4b1-1cad-48e6-8e59-04b9c64185c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:24.700054 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7c4b1-1cad-48e6-8e59-04b9c64185c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:24.700339 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7c4b1-1cad-48e6-8e59-04b9c64185c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:24.700610 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7c4b1-1cad-48e6-8e59-04b9c64185c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:24.701245 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7c4b1-1cad-48e6-8e59-04b9c64185c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:24.702824 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7c4b1-1cad-48e6-8e59-04b9c64185c4 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:24.713286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1416/2813] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:33:24.717799 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e4fe1-b2e5-48c7-9f55-f8a5f673981c admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:24.727627 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e4fe1-b2e5-48c7-9f55-f8a5f673981c admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:24.732949 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24e59f-135d-4fc3-a780-2f5000f2c182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:24.733588 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24e59f-135d-4fc3-a780-2f5000f2c182 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:24.733872 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24e59f-135d-4fc3-a780-2f5000f2c182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:24.734147 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24e59f-135d-4fc3-a780-2f5000f2c182 admin admin] PATH_INFO: `/v3/projects/a2f727762d144d47b742c68b17863927/users/e1b0321c42894d87aabab5568888e63b/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:24.748616 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24e59f-135d-4fc3-a780-2f5000f2c182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2f727762d144d47b742c68b17863927, user_id=e1b0321c42894d87aabab5568888e63b, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:24.751259 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24e59f-135d-4fc3-a780-2f5000f2c182 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:24.764568 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1398/2814] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:33:24 2022] PUT /identity/v3/projects/a2f727762d144d47b742c68b17863927/users/e1b0321c42894d87aabab5568888e63b/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:33:24.768408 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7c4b1-1cad-48e6-8e59-04b9c64185c4 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:24.778583 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7c4b1-1cad-48e6-8e59-04b9c64185c4 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:24.833436 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a48d0-2a9f-438c-abbf-7fb8a1e82deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:24.834075 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a48d0-2a9f-438c-abbf-7fb8a1e82deb admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:24.834291 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a48d0-2a9f-438c-abbf-7fb8a1e82deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:24.834562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a48d0-2a9f-438c-abbf-7fb8a1e82deb admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:24.835187 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a48d0-2a9f-438c-abbf-7fb8a1e82deb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:24.836789 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a48d0-2a9f-438c-abbf-7fb8a1e82deb admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:24.846709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1417/2815] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:33:24.850741 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24e59f-135d-4fc3-a780-2f5000f2c182 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:24.861515 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24e59f-135d-4fc3-a780-2f5000f2c182 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:24.866365 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce986b-2675-49c7-bef3-8718a59a65d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:24.867018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce986b-2675-49c7-bef3-8718a59a65d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:24.867298 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce986b-2675-49c7-bef3-8718a59a65d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:24.867572 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce986b-2675-49c7-bef3-8718a59a65d2 admin admin] PATH_INFO: `/v3/projects/a2f727762d144d47b742c68b17863927/users/e1b0321c42894d87aabab5568888e63b/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:24.869588 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce986b-2675-49c7-bef3-8718a59a65d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2f727762d144d47b742c68b17863927, user_id=e1b0321c42894d87aabab5568888e63b, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:24.872547 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce986b-2675-49c7-bef3-8718a59a65d2 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:24.882637 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1399/2816] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:33:24 2022] PUT /identity/v3/projects/a2f727762d144d47b742c68b17863927/users/e1b0321c42894d87aabab5568888e63b/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:33:24.892885 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf783f4-ae42-4214-946a-6f33ddcbeca5 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:24.893175 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf783f4-ae42-4214-946a-6f33ddcbeca5 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:24.893454 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf783f4-ae42-4214-946a-6f33ddcbeca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:24.922918 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-2cf783f4-ae42-4214-946a-6f33ddcbeca5 None None] MFA Rules not processed for user `e1b0321c42894d87aabab5568888e63b`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:33:24.924096 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2cf783f4-ae42-4214-946a-6f33ddcbeca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:24.990265 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1418/2817] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:33:24 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:33:25.001397 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce986b-2675-49c7-bef3-8718a59a65d2 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:25.003111 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4fce986b-2675-49c7-bef3-8718a59a65d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:25.060453 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce986b-2675-49c7-bef3-8718a59a65d2 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:25.064149 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f334ba-4422-4143-abd1-0646ce72d88e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:25.064742 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f334ba-4422-4143-abd1-0646ce72d88e service cinder] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:25.065023 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f334ba-4422-4143-abd1-0646ce72d88e service cinder] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:25.065297 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f334ba-4422-4143-abd1-0646ce72d88e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:25.074938 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f334ba-4422-4143-abd1-0646ce72d88e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:25.076403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f334ba-4422-4143-abd1-0646ce72d88e service cinder] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:25.086103 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1400/2818] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:33:25 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:33:25.135185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf783f4-ae42-4214-946a-6f33ddcbeca5 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:25.137105 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-2cf783f4-ae42-4214-946a-6f33ddcbeca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:25.149179 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf783f4-ae42-4214-946a-6f33ddcbeca5 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:25.153766 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb12c0-5086-4319-b09a-4c486c179811 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:25.154385 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb12c0-5086-4319-b09a-4c486c179811 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:25.154660 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb12c0-5086-4319-b09a-4c486c179811 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:25.154933 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb12c0-5086-4319-b09a-4c486c179811 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:25.195803 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:33:25.212095 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb12c0-5086-4319-b09a-4c486c179811 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:25.213509 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb12c0-5086-4319-b09a-4c486c179811 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:25.222706 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1419/2819] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:33:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:33:28.473921 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f334ba-4422-4143-abd1-0646ce72d88e service cinder] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:28.504338 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:33:28.538413 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f334ba-4422-4143-abd1-0646ce72d88e service cinder] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:28.542975 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33297fc8-1fed-470b-816b-adc47ef2791c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:28.543590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33297fc8-1fed-470b-816b-adc47ef2791c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:28.543909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33297fc8-1fed-470b-816b-adc47ef2791c admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:28.544182 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33297fc8-1fed-470b-816b-adc47ef2791c admin admin] PATH_INFO: `/v3/users/eb172b7c8ff049e1aaff449d2181ba5d` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:28.545280 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33297fc8-1fed-470b-816b-adc47ef2791c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb172b7c8ff049e1aaff449d2181ba5d)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:28.547182 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33297fc8-1fed-470b-816b-adc47ef2791c admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:28.593539 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-33297fc8-1fed-470b-816b-adc47ef2791c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb172b7c8ff049e1aaff449d2181ba5d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:28.600748 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-33297fc8-1fed-470b-816b-adc47ef2791c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb172b7c8ff049e1aaff449d2181ba5d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:28.601366 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-33297fc8-1fed-470b-816b-adc47ef2791c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb172b7c8ff049e1aaff449d2181ba5d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:28.601975 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-33297fc8-1fed-470b-816b-adc47ef2791c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb172b7c8ff049e1aaff449d2181ba5d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:28.613982 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-33297fc8-1fed-470b-816b-adc47ef2791c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb172b7c8ff049e1aaff449d2181ba5d'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:28.627983 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1401/2820] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:33:28 2022] DELETE /identity/v3/users/eb172b7c8ff049e1aaff449d2181ba5d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:33:28.632790 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d33675-58ab-4574-a9a2-f2a5eb90b665 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:28.633275 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d33675-58ab-4574-a9a2-f2a5eb90b665 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:28.633523 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d33675-58ab-4574-a9a2-f2a5eb90b665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:28.653650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-60d33675-58ab-4574-a9a2-f2a5eb90b665 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:33:28.654867 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-60d33675-58ab-4574-a9a2-f2a5eb90b665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:28.716129 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1420/2821] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:33:28.725664 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33297fc8-1fed-470b-816b-adc47ef2791c admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:28.727059 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-33297fc8-1fed-470b-816b-adc47ef2791c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:28.783433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33297fc8-1fed-470b-816b-adc47ef2791c admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:28.787218 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64167991-1a2b-4458-b279-0189113da120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:28.787856 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64167991-1a2b-4458-b279-0189113da120 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:28.788145 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64167991-1a2b-4458-b279-0189113da120 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:28.788422 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64167991-1a2b-4458-b279-0189113da120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:28.798051 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64167991-1a2b-4458-b279-0189113da120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:28.799480 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64167991-1a2b-4458-b279-0189113da120 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:28.809429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1402/2822] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:33:29.071124 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d33675-58ab-4574-a9a2-f2a5eb90b665 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:29.072783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-60d33675-58ab-4574-a9a2-f2a5eb90b665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:29.085095 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d33675-58ab-4574-a9a2-f2a5eb90b665 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:29.090048 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84794b9d-6d2e-4e06-aca5-dff059b40b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:29.090664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84794b9d-6d2e-4e06-aca5-dff059b40b38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:29.090938 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84794b9d-6d2e-4e06-aca5-dff059b40b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:29.091201 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84794b9d-6d2e-4e06-aca5-dff059b40b38 admin admin] PATH_INFO: `/v3/projects/05a2fd94b42847628da44d04c8e9f3e6` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:29.092400 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84794b9d-6d2e-4e06-aca5-dff059b40b38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05a2fd94b42847628da44d04c8e9f3e6)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:29.094367 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84794b9d-6d2e-4e06-aca5-dff059b40b38 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:29.141388 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-84794b9d-6d2e-4e06-aca5-dff059b40b38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05a2fd94b42847628da44d04c8e9f3e6', 'request_id': 'req-84794b9d-6d2e-4e06-aca5-dff059b40b38'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:29.147651 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-84794b9d-6d2e-4e06-aca5-dff059b40b38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05a2fd94b42847628da44d04c8e9f3e6', 'request_id': 'req-84794b9d-6d2e-4e06-aca5-dff059b40b38'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:29.154171 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-84794b9d-6d2e-4e06-aca5-dff059b40b38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05a2fd94b42847628da44d04c8e9f3e6', 'request_id': 'req-84794b9d-6d2e-4e06-aca5-dff059b40b38'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:29.167451 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-84794b9d-6d2e-4e06-aca5-dff059b40b38 admin admin] The token cache is being invalidate because project 05a2fd94b42847628da44d04c8e9f3e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:33:29.167736 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-84794b9d-6d2e-4e06-aca5-dff059b40b38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:29.168414 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-84794b9d-6d2e-4e06-aca5-dff059b40b38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:29.169911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1421/2823] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:33:29 2022] DELETE /identity/v3/projects/05a2fd94b42847628da44d04c8e9f3e6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:33:29.173991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64167991-1a2b-4458-b279-0189113da120 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:29.184909 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64167991-1a2b-4458-b279-0189113da120 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:29.234742 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:33:29.245591 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77d025-427a-478b-bd7b-d3966a12493b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:29.246269 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77d025-427a-478b-bd7b-d3966a12493b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:29.246508 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77d025-427a-478b-bd7b-d3966a12493b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:29.246772 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77d025-427a-478b-bd7b-d3966a12493b admin admin] PATH_INFO: `/v3/users/e1b0321c42894d87aabab5568888e63b` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:29.247927 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77d025-427a-478b-bd7b-d3966a12493b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1b0321c42894d87aabab5568888e63b)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:29.249837 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77d025-427a-478b-bd7b-d3966a12493b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:29.295293 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0d77d025-427a-478b-bd7b-d3966a12493b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1b0321c42894d87aabab5568888e63b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:29.302836 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0d77d025-427a-478b-bd7b-d3966a12493b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1b0321c42894d87aabab5568888e63b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:29.303493 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0d77d025-427a-478b-bd7b-d3966a12493b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1b0321c42894d87aabab5568888e63b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:29.304143 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0d77d025-427a-478b-bd7b-d3966a12493b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1b0321c42894d87aabab5568888e63b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:29.316141 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-0d77d025-427a-478b-bd7b-d3966a12493b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1b0321c42894d87aabab5568888e63b'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:29.328838 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1403/2824] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:33:29 2022] DELETE /identity/v3/users/e1b0321c42894d87aabab5568888e63b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:33:29.600813 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84794b9d-6d2e-4e06-aca5-dff059b40b38 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:29.602256 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-84794b9d-6d2e-4e06-aca5-dff059b40b38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:29.617329 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84794b9d-6d2e-4e06-aca5-dff059b40b38 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:29.673800 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8eea09-0785-4c8b-8284-6f6f4dac99d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:29.674423 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8eea09-0785-4c8b-8284-6f6f4dac99d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:29.674704 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8eea09-0785-4c8b-8284-6f6f4dac99d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:29.674975 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8eea09-0785-4c8b-8284-6f6f4dac99d4 admin admin] PATH_INFO: `/v3/projects/a2f727762d144d47b742c68b17863927` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:29.676125 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8eea09-0785-4c8b-8284-6f6f4dac99d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2f727762d144d47b742c68b17863927)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:29.678010 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8eea09-0785-4c8b-8284-6f6f4dac99d4 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:29.685821 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:33:29.725948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ef8eea09-0785-4c8b-8284-6f6f4dac99d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2f727762d144d47b742c68b17863927', 'request_id': 'req-ef8eea09-0785-4c8b-8284-6f6f4dac99d4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:29.732381 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ef8eea09-0785-4c8b-8284-6f6f4dac99d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2f727762d144d47b742c68b17863927', 'request_id': 'req-ef8eea09-0785-4c8b-8284-6f6f4dac99d4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:29.737967 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ef8eea09-0785-4c8b-8284-6f6f4dac99d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2f727762d144d47b742c68b17863927', 'request_id': 'req-ef8eea09-0785-4c8b-8284-6f6f4dac99d4'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:29.751501 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ef8eea09-0785-4c8b-8284-6f6f4dac99d4 admin admin] The token cache is being invalidate because project a2f727762d144d47b742c68b17863927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:33:29.751821 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ef8eea09-0785-4c8b-8284-6f6f4dac99d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:29.752452 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-ef8eea09-0785-4c8b-8284-6f6f4dac99d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:33:29.753924 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1422/2825] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:33:29 2022] DELETE /identity/v3/projects/a2f727762d144d47b742c68b17863927 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:33:29.759512 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb15746-8b29-46ef-bdd0-505b5e471ccc None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:29.760561 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb15746-8b29-46ef-bdd0-505b5e471ccc None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:29.760561 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb15746-8b29-46ef-bdd0-505b5e471ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:29.780173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-bbb15746-8b29-46ef-bdd0-505b5e471ccc None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:33:29.781292 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bbb15746-8b29-46ef-bdd0-505b5e471ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:29.840603 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1404/2826] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:33:29.848644 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f422f-8245-4352-87eb-f40cbe3b9d51 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:29.848928 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f422f-8245-4352-87eb-f40cbe3b9d51 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:29.849201 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f422f-8245-4352-87eb-f40cbe3b9d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:29.868574 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-9a8f422f-8245-4352-87eb-f40cbe3b9d51 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:33:29.870265 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-9a8f422f-8245-4352-87eb-f40cbe3b9d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:29.878093 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1423/2827] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:33:29.882160 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb15746-8b29-46ef-bdd0-505b5e471ccc None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:29.892492 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb15746-8b29-46ef-bdd0-505b5e471ccc None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:29.897026 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c088e01-bcc2-49c1-b704-ee2079f77124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:29.897611 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c088e01-bcc2-49c1-b704-ee2079f77124 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:29.897890 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c088e01-bcc2-49c1-b704-ee2079f77124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:29.898161 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c088e01-bcc2-49c1-b704-ee2079f77124 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:29.898808 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c088e01-bcc2-49c1-b704-ee2079f77124 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:33:29.899222 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c088e01-bcc2-49c1-b704-ee2079f77124 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:29.900908 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c088e01-bcc2-49c1-b704-ee2079f77124 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:29.912613 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1405/2828] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:33:29.917006 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f422f-8245-4352-87eb-f40cbe3b9d51 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:29.926601 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f422f-8245-4352-87eb-f40cbe3b9d51 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:29.931326 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ca345-cfe1-4ad8-88a5-ee43310c0af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:29.932455 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ca345-cfe1-4ad8-88a5-ee43310c0af7 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:29.932738 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ca345-cfe1-4ad8-88a5-ee43310c0af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:29.933003 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ca345-cfe1-4ad8-88a5-ee43310c0af7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:29.933688 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ca345-cfe1-4ad8-88a5-ee43310c0af7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:29.935895 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ca345-cfe1-4ad8-88a5-ee43310c0af7 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:29.963166 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1424/2829] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:33:29 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:33:29.967294 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c088e01-bcc2-49c1-b704-ee2079f77124 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:29.977670 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c088e01-bcc2-49c1-b704-ee2079f77124 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:30.027823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:33:30.034615 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b9275-3f85-4439-8c3a-ec3bd6246805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:30.035233 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b9275-3f85-4439-8c3a-ec3bd6246805 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:30.035528 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b9275-3f85-4439-8c3a-ec3bd6246805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:30.036044 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b9275-3f85-4439-8c3a-ec3bd6246805 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:30.036995 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b9275-3f85-4439-8c3a-ec3bd6246805 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:30.039037 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b9275-3f85-4439-8c3a-ec3bd6246805 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:30.059913 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1406/2830] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:33:29 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:33:30.064030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ca345-cfe1-4ad8-88a5-ee43310c0af7 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:30.075023 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ca345-cfe1-4ad8-88a5-ee43310c0af7 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:30.079604 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26788311-38aa-410f-b66c-bb9c56a21762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:30.080229 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26788311-38aa-410f-b66c-bb9c56a21762 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:30.080506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26788311-38aa-410f-b66c-bb9c56a21762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:30.080776 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26788311-38aa-410f-b66c-bb9c56a21762 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:30.081415 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26788311-38aa-410f-b66c-bb9c56a21762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:30.082981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26788311-38aa-410f-b66c-bb9c56a21762 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:30.092771 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1425/2831] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:33:30.096534 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b9275-3f85-4439-8c3a-ec3bd6246805 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:30.106441 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b9275-3f85-4439-8c3a-ec3bd6246805 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:30.111020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb48666-193a-4d6a-8f7c-42d4e15db849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:30.111608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb48666-193a-4d6a-8f7c-42d4e15db849 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:30.111953 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb48666-193a-4d6a-8f7c-42d4e15db849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:30.112228 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb48666-193a-4d6a-8f7c-42d4e15db849 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:30.112868 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb48666-193a-4d6a-8f7c-42d4e15db849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:30.114449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb48666-193a-4d6a-8f7c-42d4e15db849 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:30.127618 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1407/2832] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:33:30.132900 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26788311-38aa-410f-b66c-bb9c56a21762 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:30.143854 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26788311-38aa-410f-b66c-bb9c56a21762 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:30.148559 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d1587-9a2e-4cb6-b2c3-0000e96b3bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:30.149198 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d1587-9a2e-4cb6-b2c3-0000e96b3bc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:30.149481 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d1587-9a2e-4cb6-b2c3-0000e96b3bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:30.149754 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d1587-9a2e-4cb6-b2c3-0000e96b3bc8 admin admin] PATH_INFO: `/v3/projects/914cee0eaeb1460db8b0c068920a962c/users/cac8337d54a24d0c8fa4c836557bf933/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:30.166748 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d1587-9a2e-4cb6-b2c3-0000e96b3bc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=914cee0eaeb1460db8b0c068920a962c, user_id=cac8337d54a24d0c8fa4c836557bf933, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:30.170145 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d1587-9a2e-4cb6-b2c3-0000e96b3bc8 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:30.183223 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1426/2833] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:33:30 2022] PUT /identity/v3/projects/914cee0eaeb1460db8b0c068920a962c/users/cac8337d54a24d0c8fa4c836557bf933/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:33:30.188254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7f5af-7720-4641-9c6a-a48dbe634af3 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:30.188546 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7f5af-7720-4641-9c6a-a48dbe634af3 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:30.188817 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7f5af-7720-4641-9c6a-a48dbe634af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:30.208037 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-09d7f5af-7720-4641-9c6a-a48dbe634af3 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:33:30.209310 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-09d7f5af-7720-4641-9c6a-a48dbe634af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:30.266955 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1408/2834] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:33:30.276752 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d1587-9a2e-4cb6-b2c3-0000e96b3bc8 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:30.278442 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-7b1d1587-9a2e-4cb6-b2c3-0000e96b3bc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:30.334555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d1587-9a2e-4cb6-b2c3-0000e96b3bc8 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:30.338152 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cdddf-696f-4f58-be46-d5c0057832af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:30.338750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cdddf-696f-4f58-be46-d5c0057832af service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:30.339032 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cdddf-696f-4f58-be46-d5c0057832af service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:30.339308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cdddf-696f-4f58-be46-d5c0057832af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:30.348427 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cdddf-696f-4f58-be46-d5c0057832af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:30.349843 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cdddf-696f-4f58-be46-d5c0057832af service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:30.359620 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1427/2835] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:33:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:33:31.046706 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d581cc-af78-46df-9146-a7e4f1aca64d None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:31.047717 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d581cc-af78-46df-9146-a7e4f1aca64d None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:31.048031 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d581cc-af78-46df-9146-a7e4f1aca64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:31.076231 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-87d581cc-af78-46df-9146-a7e4f1aca64d None None] MFA Rules not processed for user `cac8337d54a24d0c8fa4c836557bf933`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:33:31.077445 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-87d581cc-af78-46df-9146-a7e4f1aca64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:31.145172 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1409/2836] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:33:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:33:31.156747 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cdddf-696f-4f58-be46-d5c0057832af service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:31.158330 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-de2cdddf-696f-4f58-be46-d5c0057832af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:31.201111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:33:31.221902 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cdddf-696f-4f58-be46-d5c0057832af service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:31.225481 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f0fa2d-89b1-4d31-a9b6-4d495255fdb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:31.226103 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0fa2d-89b1-4d31-a9b6-4d495255fdb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:31.226378 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0fa2d-89b1-4d31-a9b6-4d495255fdb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:31.226646 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0fa2d-89b1-4d31-a9b6-4d495255fdb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:31.235068 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f0fa2d-89b1-4d31-a9b6-4d495255fdb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:31.236562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f0fa2d-89b1-4d31-a9b6-4d495255fdb7 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:31.245572 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1428/2837] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:33:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:33:32.598678 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c8f94-5af3-4c67-a19d-2f2c9f01f4a2 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:32.599203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c8f94-5af3-4c67-a19d-2f2c9f01f4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:32.599203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c8f94-5af3-4c67-a19d-2f2c9f01f4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:32.617818 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-ae5c8f94-5af3-4c67-a19d-2f2c9f01f4a2 None None] MFA Rules not processed for user `cac8337d54a24d0c8fa4c836557bf933`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:33:32.619018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ae5c8f94-5af3-4c67-a19d-2f2c9f01f4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:32.625859 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1410/2838] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:33:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:33:32.664435 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f0fa2d-89b1-4d31-a9b6-4d495255fdb7 service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:32.665803 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-28f0fa2d-89b1-4d31-a9b6-4d495255fdb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:32.721214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f0fa2d-89b1-4d31-a9b6-4d495255fdb7 service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:32.724780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f90ec-85ff-47c1-87ae-b57f821892e8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88670b0e1a0f460d8f74917c92db9b9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:32.725367 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f90ec-85ff-47c1-87ae-b57f821892e8 service glance] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:32.725643 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f90ec-85ff-47c1-87ae-b57f821892e8 service glance] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:32.725911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f90ec-85ff-47c1-87ae-b57f821892e8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:32.727581 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-aa7f90ec-85ff-47c1-87ae-b57f821892e8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:32.794712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f90ec-85ff-47c1-87ae-b57f821892e8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:32.796067 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f90ec-85ff-47c1-87ae-b57f821892e8 service glance] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:32.814193 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1429/2839] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:33:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:33:33.340933 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c8f94-5af3-4c67-a19d-2f2c9f01f4a2 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:33.342502 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-ae5c8f94-5af3-4c67-a19d-2f2c9f01f4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:33.354755 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c8f94-5af3-4c67-a19d-2f2c9f01f4a2 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:33.358253 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dfdddf-b01b-409f-bbf5-73cb6d1f400a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:33.358842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfdddf-b01b-409f-bbf5-73cb6d1f400a service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:33.359112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfdddf-b01b-409f-bbf5-73cb6d1f400a service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:33.359378 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfdddf-b01b-409f-bbf5-73cb6d1f400a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:33.361047 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-47dfdddf-b01b-409f-bbf5-73cb6d1f400a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:33.374858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dfdddf-b01b-409f-bbf5-73cb6d1f400a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:33.376416 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dfdddf-b01b-409f-bbf5-73cb6d1f400a service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:33.388491 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1411/2840] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:33:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:33:42.807055 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=64 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:33:43.382716 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=76,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:33:48.870531 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4c859-02f0-44ee-8570-587b48f70b6f None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:48.871050 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4c859-02f0-44ee-8570-587b48f70b6f None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:48.871311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4c859-02f0-44ee-8570-587b48f70b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:48.896981 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-efc4c859-02f0-44ee-8570-587b48f70b6f None None] MFA Rules not processed for user `cac8337d54a24d0c8fa4c836557bf933`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:33:48.898206 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-efc4c859-02f0-44ee-8570-587b48f70b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:48.905220 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1430/2841] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:33:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:33:51.996347 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dfdddf-b01b-409f-bbf5-73cb6d1f400a service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:52.008097 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dfdddf-b01b-409f-bbf5-73cb6d1f400a service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:52.012659 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e72f39-9b87-4adc-a718-3568958a0c1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:52.013464 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e72f39-9b87-4adc-a718-3568958a0c1b service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:52.013923 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e72f39-9b87-4adc-a718-3568958a0c1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:52.014378 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e72f39-9b87-4adc-a718-3568958a0c1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:52.017012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-54e72f39-9b87-4adc-a718-3568958a0c1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:52.031688 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e72f39-9b87-4adc-a718-3568958a0c1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:52.033569 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e72f39-9b87-4adc-a718-3568958a0c1b service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:52.044806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1412/2842] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:33:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:33:57.961062 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4c859-02f0-44ee-8570-587b48f70b6f None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:33:57.971460 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4c859-02f0-44ee-8570-587b48f70b6f None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:33:57.975146 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71eba7d-ee9a-4ded-bf10-078da3072674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:33:57.975737 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71eba7d-ee9a-4ded-bf10-078da3072674 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:33:57.976047 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71eba7d-ee9a-4ded-bf10-078da3072674 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:33:57.976317 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71eba7d-ee9a-4ded-bf10-078da3072674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:33:57.978100 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c71eba7d-ee9a-4ded-bf10-078da3072674 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:33:57.989865 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71eba7d-ee9a-4ded-bf10-078da3072674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:33:57.991258 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71eba7d-ee9a-4ded-bf10-078da3072674 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:33:58.000830 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1431/2843] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:33:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:34:02.038964 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=6 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:34:04.466436 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e72f39-9b87-4adc-a718-3568958a0c1b service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:34:04.468011 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-54e72f39-9b87-4adc-a718-3568958a0c1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:34:04.483230 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e72f39-9b87-4adc-a718-3568958a0c1b service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:34:04.487936 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fdd63-d0bb-4f5f-a80d-776023c39268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:34:04.488424 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fdd63-d0bb-4f5f-a80d-776023c39268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:04.488705 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fdd63-d0bb-4f5f-a80d-776023c39268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:04.488979 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fdd63-d0bb-4f5f-a80d-776023c39268 admin admin] PATH_INFO: `/v3/users/7f546dc2d8b24eeea8c6eac968c56806` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:04.490103 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fdd63-d0bb-4f5f-a80d-776023c39268 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f546dc2d8b24eeea8c6eac968c56806)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:34:04.492066 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fdd63-d0bb-4f5f-a80d-776023c39268 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:34:04.539053 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c56fdd63-d0bb-4f5f-a80d-776023c39268 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f546dc2d8b24eeea8c6eac968c56806'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:04.546685 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c56fdd63-d0bb-4f5f-a80d-776023c39268 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f546dc2d8b24eeea8c6eac968c56806'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:04.547368 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c56fdd63-d0bb-4f5f-a80d-776023c39268 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f546dc2d8b24eeea8c6eac968c56806'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:04.548012 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c56fdd63-d0bb-4f5f-a80d-776023c39268 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f546dc2d8b24eeea8c6eac968c56806'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:04.560652 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-c56fdd63-d0bb-4f5f-a80d-776023c39268 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f546dc2d8b24eeea8c6eac968c56806'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:04.573472 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1413/2844] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:34:04 2022] DELETE /identity/v3/users/7f546dc2d8b24eeea8c6eac968c56806 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:34:04.740369 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71eba7d-ee9a-4ded-bf10-078da3072674 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:34:04.752846 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71eba7d-ee9a-4ded-bf10-078da3072674 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:34:04.811985 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1703fc2-1269-467e-b3b1-d46d94ed9913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:34:04.812600 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1703fc2-1269-467e-b3b1-d46d94ed9913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:04.812873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1703fc2-1269-467e-b3b1-d46d94ed9913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:04.813136 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1703fc2-1269-467e-b3b1-d46d94ed9913 admin admin] PATH_INFO: `/v3/projects/8351993a1ab94f82bc238a23b2d542d3` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:04.814262 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1703fc2-1269-467e-b3b1-d46d94ed9913 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8351993a1ab94f82bc238a23b2d542d3)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:34:04.816162 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1703fc2-1269-467e-b3b1-d46d94ed9913 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:34:04.861250 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-f1703fc2-1269-467e-b3b1-d46d94ed9913 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8351993a1ab94f82bc238a23b2d542d3', 'request_id': 'req-f1703fc2-1269-467e-b3b1-d46d94ed9913'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:04.867375 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-f1703fc2-1269-467e-b3b1-d46d94ed9913 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8351993a1ab94f82bc238a23b2d542d3', 'request_id': 'req-f1703fc2-1269-467e-b3b1-d46d94ed9913'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:04.873636 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-f1703fc2-1269-467e-b3b1-d46d94ed9913 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8351993a1ab94f82bc238a23b2d542d3', 'request_id': 'req-f1703fc2-1269-467e-b3b1-d46d94ed9913'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:04.885296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-f1703fc2-1269-467e-b3b1-d46d94ed9913 admin admin] The token cache is being invalidate because project 8351993a1ab94f82bc238a23b2d542d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:34:04.885585 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-f1703fc2-1269-467e-b3b1-d46d94ed9913 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:04.886217 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-f1703fc2-1269-467e-b3b1-d46d94ed9913 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:04.887675 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1432/2845] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:34:04 2022] DELETE /identity/v3/projects/8351993a1ab94f82bc238a23b2d542d3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:34:04.891702 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fdd63-d0bb-4f5f-a80d-776023c39268 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:34:04.893146 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-c56fdd63-d0bb-4f5f-a80d-776023c39268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:34:04.906229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fdd63-d0bb-4f5f-a80d-776023c39268 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:34:04.963166 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fd756-ee54-4c7b-8ac0-0f6bce151123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:34:04.963723 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fd756-ee54-4c7b-8ac0-0f6bce151123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:04.964033 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fd756-ee54-4c7b-8ac0-0f6bce151123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:04.964308 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fd756-ee54-4c7b-8ac0-0f6bce151123 admin admin] PATH_INFO: `/v3/users/4e87385147af409eb2bde82b870c540e` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:04.965439 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52fd756-ee54-4c7b-8ac0-0f6bce151123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e87385147af409eb2bde82b870c540e)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:34:04.967355 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52fd756-ee54-4c7b-8ac0-0f6bce151123 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:34:05.012426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e52fd756-ee54-4c7b-8ac0-0f6bce151123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e87385147af409eb2bde82b870c540e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:05.019967 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e52fd756-ee54-4c7b-8ac0-0f6bce151123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e87385147af409eb2bde82b870c540e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:05.020608 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e52fd756-ee54-4c7b-8ac0-0f6bce151123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e87385147af409eb2bde82b870c540e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:05.021363 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e52fd756-ee54-4c7b-8ac0-0f6bce151123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e87385147af409eb2bde82b870c540e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:05.033748 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-e52fd756-ee54-4c7b-8ac0-0f6bce151123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e87385147af409eb2bde82b870c540e'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:05.035694 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:34:05.048987 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1414/2846] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:34:04 2022] DELETE /identity/v3/users/4e87385147af409eb2bde82b870c540e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:34:05.221305 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1703fc2-1269-467e-b3b1-d46d94ed9913 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:34:05.222792 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-f1703fc2-1269-467e-b3b1-d46d94ed9913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:34:05.238264 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1703fc2-1269-467e-b3b1-d46d94ed9913 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:34:05.295185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a98c4-1c8f-4f17-afad-64d3c49c77e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:34:05.295831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a98c4-1c8f-4f17-afad-64d3c49c77e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:05.296115 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a98c4-1c8f-4f17-afad-64d3c49c77e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:05.296386 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a98c4-1c8f-4f17-afad-64d3c49c77e5 admin admin] PATH_INFO: `/v3/projects/45c46d122e74432a8228d27ed8b6f876` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:05.297520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a98c4-1c8f-4f17-afad-64d3c49c77e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45c46d122e74432a8228d27ed8b6f876)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:34:05.299406 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a98c4-1c8f-4f17-afad-64d3c49c77e5 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:34:05.310909 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:34:05.347730 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-de0a98c4-1c8f-4f17-afad-64d3c49c77e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45c46d122e74432a8228d27ed8b6f876', 'request_id': 'req-de0a98c4-1c8f-4f17-afad-64d3c49c77e5'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:05.354688 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-de0a98c4-1c8f-4f17-afad-64d3c49c77e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45c46d122e74432a8228d27ed8b6f876', 'request_id': 'req-de0a98c4-1c8f-4f17-afad-64d3c49c77e5'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:05.365536 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-de0a98c4-1c8f-4f17-afad-64d3c49c77e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45c46d122e74432a8228d27ed8b6f876', 'request_id': 'req-de0a98c4-1c8f-4f17-afad-64d3c49c77e5'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:05.378205 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-de0a98c4-1c8f-4f17-afad-64d3c49c77e5 admin admin] The token cache is being invalidate because project 45c46d122e74432a8228d27ed8b6f876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:34:05.378499 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-de0a98c4-1c8f-4f17-afad-64d3c49c77e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:05.379172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-de0a98c4-1c8f-4f17-afad-64d3c49c77e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:05.380712 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1433/2847] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:34:05 2022] DELETE /identity/v3/projects/45c46d122e74432a8228d27ed8b6f876 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:34:09.880435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987869ab-63d1-449e-8be6-d5728f1284aa None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:09.880914 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987869ab-63d1-449e-8be6-d5728f1284aa None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:09.881016 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987869ab-63d1-449e-8be6-d5728f1284aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:09.939138 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-987869ab-63d1-449e-8be6-d5728f1284aa None None] MFA Rules not processed for user `06d279b488d148428b7a3c35a9786ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:34:09.940408 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-987869ab-63d1-449e-8be6-d5728f1284aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:34:10.010458 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1415/2848] 10.222.0.81 () {58 vars in 1009 bytes} [Mon Sep 19 11:34:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:34:10.056172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a98c4-1c8f-4f17-afad-64d3c49c77e5 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:34:10.057670 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-de0a98c4-1c8f-4f17-afad-64d3c49c77e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:34:10.147710 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a98c4-1c8f-4f17-afad-64d3c49c77e5 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:34:10.153296 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1af1c6-e97c-4f9c-b766-a57bcc4c3540 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8a9732807d4921a41a6a2d5019e2f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:34:10.153893 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1af1c6-e97c-4f9c-b766-a57bcc4c3540 service swift] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:10.154174 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1af1c6-e97c-4f9c-b766-a57bcc4c3540 service swift] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:10.154445 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1af1c6-e97c-4f9c-b766-a57bcc4c3540 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:10.163213 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1af1c6-e97c-4f9c-b766-a57bcc4c3540 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:34:10.165364 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1af1c6-e97c-4f9c-b766-a57bcc4c3540 service swift] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:34:10.174894 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1434/2849] 10.222.0.81 () {60 vars in 1384 bytes} [Mon Sep 19 11:34:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 11:34:20.007112 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=48 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:34:20.171311 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:34:24.677925 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527e5e8-e4a4-465e-9d88-c700a430e711 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:24.678359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527e5e8-e4a4-465e-9d88-c700a430e711 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:24.678511 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527e5e8-e4a4-465e-9d88-c700a430e711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:24.711576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-2527e5e8-e4a4-465e-9d88-c700a430e711 None None] MFA Rules not processed for user `06d279b488d148428b7a3c35a9786ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:34:24.712841 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2527e5e8-e4a4-465e-9d88-c700a430e711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:34:24.721074 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1416/2850] 10.222.0.81 () {58 vars in 967 bytes} [Mon Sep 19 11:34:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:34:24.733579 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1af1c6-e97c-4f9c-b766-a57bcc4c3540 service swift] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:34:24.746443 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1af1c6-e97c-4f9c-b766-a57bcc4c3540 service swift] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:34:24.751863 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8b5c9-b040-4cf1-9807-127b0f2f4a38 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8a9732807d4921a41a6a2d5019e2f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:34:24.752451 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8b5c9-b040-4cf1-9807-127b0f2f4a38 service swift] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:24.752789 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8b5c9-b040-4cf1-9807-127b0f2f4a38 service swift] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:24.753058 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8b5c9-b040-4cf1-9807-127b0f2f4a38 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:24.761855 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8b5c9-b040-4cf1-9807-127b0f2f4a38 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:34:24.763394 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8b5c9-b040-4cf1-9807-127b0f2f4a38 service swift] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:34:24.771530 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1435/2851] 10.222.0.81 () {60 vars in 1384 bytes} [Mon Sep 19 11:34:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:34:26.914375 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527e5e8-e4a4-465e-9d88-c700a430e711 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:34:26.916035 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2527e5e8-e4a4-465e-9d88-c700a430e711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:34:26.984847 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527e5e8-e4a4-465e-9d88-c700a430e711 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:34:26.989695 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a6e1d-bcc9-478a-8dd6-5d13c5a4664a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:34:26.990326 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a6e1d-bcc9-478a-8dd6-5d13c5a4664a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:26.990609 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a6e1d-bcc9-478a-8dd6-5d13c5a4664a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:26.990886 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a6e1d-bcc9-478a-8dd6-5d13c5a4664a admin admin] PATH_INFO: `/v3/users/cac8337d54a24d0c8fa4c836557bf933` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:26.992572 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a6e1d-bcc9-478a-8dd6-5d13c5a4664a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cac8337d54a24d0c8fa4c836557bf933)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:34:26.994492 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a6e1d-bcc9-478a-8dd6-5d13c5a4664a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:34:27.038488 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-795a6e1d-bcc9-478a-8dd6-5d13c5a4664a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cac8337d54a24d0c8fa4c836557bf933'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:27.045175 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-795a6e1d-bcc9-478a-8dd6-5d13c5a4664a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cac8337d54a24d0c8fa4c836557bf933'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:27.046053 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-795a6e1d-bcc9-478a-8dd6-5d13c5a4664a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cac8337d54a24d0c8fa4c836557bf933'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:27.046704 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-795a6e1d-bcc9-478a-8dd6-5d13c5a4664a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cac8337d54a24d0c8fa4c836557bf933'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:27.058835 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-795a6e1d-bcc9-478a-8dd6-5d13c5a4664a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cac8337d54a24d0c8fa4c836557bf933'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:27.071449 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1417/2852] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:34:26 2022] DELETE /identity/v3/users/cac8337d54a24d0c8fa4c836557bf933 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:34:27.241045 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8b5c9-b040-4cf1-9807-127b0f2f4a38 service swift] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:34:27.252127 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8b5c9-b040-4cf1-9807-127b0f2f4a38 service swift] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:34:27.313308 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04b6b6-6ea1-4f14-9969-2a6925550c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:34:27.313932 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04b6b6-6ea1-4f14-9969-2a6925550c5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:27.314214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04b6b6-6ea1-4f14-9969-2a6925550c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:27.314492 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04b6b6-6ea1-4f14-9969-2a6925550c5d admin admin] PATH_INFO: `/v3/projects/914cee0eaeb1460db8b0c068920a962c` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:27.315602 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04b6b6-6ea1-4f14-9969-2a6925550c5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=914cee0eaeb1460db8b0c068920a962c)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:34:27.317548 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04b6b6-6ea1-4f14-9969-2a6925550c5d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:34:27.362512 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5d04b6b6-6ea1-4f14-9969-2a6925550c5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '914cee0eaeb1460db8b0c068920a962c', 'request_id': 'req-5d04b6b6-6ea1-4f14-9969-2a6925550c5d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:27.369343 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5d04b6b6-6ea1-4f14-9969-2a6925550c5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '914cee0eaeb1460db8b0c068920a962c', 'request_id': 'req-5d04b6b6-6ea1-4f14-9969-2a6925550c5d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:27.374881 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5d04b6b6-6ea1-4f14-9969-2a6925550c5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '914cee0eaeb1460db8b0c068920a962c', 'request_id': 'req-5d04b6b6-6ea1-4f14-9969-2a6925550c5d'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:27.387191 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5d04b6b6-6ea1-4f14-9969-2a6925550c5d admin admin] The token cache is being invalidate because project 914cee0eaeb1460db8b0c068920a962c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:34:27.387483 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5d04b6b6-6ea1-4f14-9969-2a6925550c5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:27.388111 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-5d04b6b6-6ea1-4f14-9969-2a6925550c5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:34:27.389625 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1436/2853] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:34:27 2022] DELETE /identity/v3/projects/914cee0eaeb1460db8b0c068920a962c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:34:27.394560 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6e1c3-6609-4ff3-b2f3-f9828f632d41 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:27.394858 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6e1c3-6609-4ff3-b2f3-f9828f632d41 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:27.395125 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6e1c3-6609-4ff3-b2f3-f9828f632d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:27.415901 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f5a6e1c3-6609-4ff3-b2f3-f9828f632d41 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:34:27.417147 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f5a6e1c3-6609-4ff3-b2f3-f9828f632d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:34:27.474827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1418/2854] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:34:27.483170 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7437d6-8805-45c8-a27e-9b646da867d8 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:27.483453 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7437d6-8805-45c8-a27e-9b646da867d8 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:27.483724 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7437d6-8805-45c8-a27e-9b646da867d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:27.503654 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-3d7437d6-8805-45c8-a27e-9b646da867d8 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:34:27.504869 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3d7437d6-8805-45c8-a27e-9b646da867d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:34:27.512596 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1437/2855] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:34:27.516686 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6e1c3-6609-4ff3-b2f3-f9828f632d41 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:34:27.526786 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6e1c3-6609-4ff3-b2f3-f9828f632d41 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:34:27.531277 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59e4ff-3fe1-423f-a0d5-76985ef3b69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:34:27.531899 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59e4ff-3fe1-423f-a0d5-76985ef3b69a admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:27.532181 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59e4ff-3fe1-423f-a0d5-76985ef3b69a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:27.532452 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59e4ff-3fe1-423f-a0d5-76985ef3b69a admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:27.533095 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59e4ff-3fe1-423f-a0d5-76985ef3b69a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:34:27.533422 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59e4ff-3fe1-423f-a0d5-76985ef3b69a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:34:27.535013 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59e4ff-3fe1-423f-a0d5-76985ef3b69a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:34:27.544150 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:34:27.547970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1419/2856] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:34:27.552502 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7437d6-8805-45c8-a27e-9b646da867d8 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:34:27.561872 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7437d6-8805-45c8-a27e-9b646da867d8 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:34:27.566528 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506affe5-e693-42dd-8472-e44116d12892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:34:27.567122 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506affe5-e693-42dd-8472-e44116d12892 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:27.567402 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506affe5-e693-42dd-8472-e44116d12892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:27.567674 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506affe5-e693-42dd-8472-e44116d12892 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:27.568344 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506affe5-e693-42dd-8472-e44116d12892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:34:27.570489 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506affe5-e693-42dd-8472-e44116d12892 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:34:27.598184 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1438/2857] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:34:27 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:34:27.602793 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59e4ff-3fe1-423f-a0d5-76985ef3b69a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:34:27.612619 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59e4ff-3fe1-423f-a0d5-76985ef3b69a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:34:27.667992 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5f2fb-355e-4bb8-a7ce-4278f811912f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:34:27.668591 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5f2fb-355e-4bb8-a7ce-4278f811912f admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:27.668866 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5f2fb-355e-4bb8-a7ce-4278f811912f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:27.669133 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5f2fb-355e-4bb8-a7ce-4278f811912f admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:27.669769 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5f2fb-355e-4bb8-a7ce-4278f811912f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:34:27.671661 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5f2fb-355e-4bb8-a7ce-4278f811912f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:34:27.692086 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1420/2858] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:34:27 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:34:27.696459 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506affe5-e693-42dd-8472-e44116d12892 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:34:27.706571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506affe5-e693-42dd-8472-e44116d12892 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:34:27.711685 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe41a25-fc89-4391-8957-740903a5e0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:34:27.712297 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe41a25-fc89-4391-8957-740903a5e0aa admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:27.712571 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe41a25-fc89-4391-8957-740903a5e0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:27.712840 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe41a25-fc89-4391-8957-740903a5e0aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:27.713468 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe41a25-fc89-4391-8957-740903a5e0aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:34:27.715021 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe41a25-fc89-4391-8957-740903a5e0aa admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:34:27.724579 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1439/2859] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:34:27.728851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5f2fb-355e-4bb8-a7ce-4278f811912f admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:34:27.738654 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5f2fb-355e-4bb8-a7ce-4278f811912f admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:34:27.743285 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2561b-e8cd-4814-ba64-873f7f99958b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:34:27.743907 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2561b-e8cd-4814-ba64-873f7f99958b admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:27.744187 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2561b-e8cd-4814-ba64-873f7f99958b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:27.744459 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2561b-e8cd-4814-ba64-873f7f99958b admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:27.745126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de2561b-e8cd-4814-ba64-873f7f99958b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:34:27.746704 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de2561b-e8cd-4814-ba64-873f7f99958b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:34:27.757251 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1421/2860] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:34:27.761268 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe41a25-fc89-4391-8957-740903a5e0aa admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:34:27.771063 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe41a25-fc89-4391-8957-740903a5e0aa admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:34:27.776127 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d780719-4972-453f-a6eb-45ca7cc5cc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:34:27.776753 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d780719-4972-453f-a6eb-45ca7cc5cc26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:27.777029 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d780719-4972-453f-a6eb-45ca7cc5cc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:27.777298 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d780719-4972-453f-a6eb-45ca7cc5cc26 admin admin] PATH_INFO: `/v3/projects/c1a839cda0114b38833e1be9e746515b/users/cb23953bd4a54d2cb094a835a2413a8f/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:27.792012 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d780719-4972-453f-a6eb-45ca7cc5cc26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1a839cda0114b38833e1be9e746515b, user_id=cb23953bd4a54d2cb094a835a2413a8f, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:34:27.794620 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d780719-4972-453f-a6eb-45ca7cc5cc26 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:34:27.806650 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1440/2861] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:34:27 2022] PUT /identity/v3/projects/c1a839cda0114b38833e1be9e746515b/users/cb23953bd4a54d2cb094a835a2413a8f/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:34:27.817426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361be6dc-635d-459f-9904-6b3ed5453383 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:27.817712 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361be6dc-635d-459f-9904-6b3ed5453383 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:27.817986 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361be6dc-635d-459f-9904-6b3ed5453383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:27.847655 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-361be6dc-635d-459f-9904-6b3ed5453383 None None] MFA Rules not processed for user `cb23953bd4a54d2cb094a835a2413a8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:34:27.848887 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-361be6dc-635d-459f-9904-6b3ed5453383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:34:27.912605 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1422/2862] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:34:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:34:27.924341 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d780719-4972-453f-a6eb-45ca7cc5cc26 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:34:27.926039 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-4d780719-4972-453f-a6eb-45ca7cc5cc26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:34:27.941865 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:34:27.986854 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d780719-4972-453f-a6eb-45ca7cc5cc26 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:34:27.990408 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157de273-5e00-4bf7-b096-3381aad160d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:34:27.990999 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157de273-5e00-4bf7-b096-3381aad160d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:34:27.991278 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157de273-5e00-4bf7-b096-3381aad160d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:34:27.991550 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157de273-5e00-4bf7-b096-3381aad160d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:34:27.999720 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157de273-5e00-4bf7-b096-3381aad160d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:34:28.001156 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157de273-5e00-4bf7-b096-3381aad160d5 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:34:28.010353 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1441/2863] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:34:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:34:37.909748 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:34:38.005801 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=20 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:35:01.956472 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361be6dc-635d-459f-9904-6b3ed5453383 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:02.017934 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361be6dc-635d-459f-9904-6b3ed5453383 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:02.022446 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159929fa-be8d-4870-bd7d-91505611be37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:02.023587 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159929fa-be8d-4870-bd7d-91505611be37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:02.023898 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159929fa-be8d-4870-bd7d-91505611be37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:02.024203 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159929fa-be8d-4870-bd7d-91505611be37 admin admin] PATH_INFO: `/v3/users/cb23953bd4a54d2cb094a835a2413a8f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:02.025347 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159929fa-be8d-4870-bd7d-91505611be37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb23953bd4a54d2cb094a835a2413a8f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:02.027273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159929fa-be8d-4870-bd7d-91505611be37 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:02.072061 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-159929fa-be8d-4870-bd7d-91505611be37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb23953bd4a54d2cb094a835a2413a8f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:35:02.079357 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-159929fa-be8d-4870-bd7d-91505611be37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb23953bd4a54d2cb094a835a2413a8f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:35:02.079999 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-159929fa-be8d-4870-bd7d-91505611be37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb23953bd4a54d2cb094a835a2413a8f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:35:02.080611 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-159929fa-be8d-4870-bd7d-91505611be37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb23953bd4a54d2cb094a835a2413a8f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:35:02.092970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-159929fa-be8d-4870-bd7d-91505611be37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb23953bd4a54d2cb094a835a2413a8f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:35:02.105373 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1423/2864] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:35:01 2022] DELETE /identity/v3/users/cb23953bd4a54d2cb094a835a2413a8f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:35:02.109882 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e87546-179c-42d1-ab00-5d321537dca9 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:02.110365 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e87546-179c-42d1-ab00-5d321537dca9 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:02.110473 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e87546-179c-42d1-ab00-5d321537dca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:02.130511 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-d7e87546-179c-42d1-ab00-5d321537dca9 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:35:02.131706 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-d7e87546-179c-42d1-ab00-5d321537dca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:02.189605 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1442/2865] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 11:35:02.199873 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159929fa-be8d-4870-bd7d-91505611be37 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:02.201269 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-159929fa-be8d-4870-bd7d-91505611be37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:02.256426 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159929fa-be8d-4870-bd7d-91505611be37 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:02.260053 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b89ea-7d43-4146-9738-dcd5ef036b16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:02.260641 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b89ea-7d43-4146-9738-dcd5ef036b16 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:02.260916 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b89ea-7d43-4146-9738-dcd5ef036b16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:02.261184 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b89ea-7d43-4146-9738-dcd5ef036b16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:02.269849 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b89ea-7d43-4146-9738-dcd5ef036b16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:02.271261 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b89ea-7d43-4146-9738-dcd5ef036b16 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:02.280958 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1424/2866] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:35:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:35:02.547117 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e87546-179c-42d1-ab00-5d321537dca9 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:02.548609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-d7e87546-179c-42d1-ab00-5d321537dca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:02.560626 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e87546-179c-42d1-ab00-5d321537dca9 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:02.565619 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8809640-e2c4-4a13-92fa-6f2db1718e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:02.566354 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8809640-e2c4-4a13-92fa-6f2db1718e3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:02.566637 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8809640-e2c4-4a13-92fa-6f2db1718e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:02.566911 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8809640-e2c4-4a13-92fa-6f2db1718e3f admin admin] PATH_INFO: `/v3/projects/c1a839cda0114b38833e1be9e746515b` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:02.569083 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8809640-e2c4-4a13-92fa-6f2db1718e3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1a839cda0114b38833e1be9e746515b)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:02.571562 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8809640-e2c4-4a13-92fa-6f2db1718e3f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:02.618722 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e8809640-e2c4-4a13-92fa-6f2db1718e3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1a839cda0114b38833e1be9e746515b', 'request_id': 'req-e8809640-e2c4-4a13-92fa-6f2db1718e3f'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:35:02.624878 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e8809640-e2c4-4a13-92fa-6f2db1718e3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1a839cda0114b38833e1be9e746515b', 'request_id': 'req-e8809640-e2c4-4a13-92fa-6f2db1718e3f'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:35:02.630410 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e8809640-e2c4-4a13-92fa-6f2db1718e3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1a839cda0114b38833e1be9e746515b', 'request_id': 'req-e8809640-e2c4-4a13-92fa-6f2db1718e3f'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:35:02.642705 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e8809640-e2c4-4a13-92fa-6f2db1718e3f admin admin] The token cache is being invalidate because project c1a839cda0114b38833e1be9e746515b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:35:02.642989 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e8809640-e2c4-4a13-92fa-6f2db1718e3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:35:02.643630 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-e8809640-e2c4-4a13-92fa-6f2db1718e3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:35:02.645100 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1443/2867] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:35:02 2022] DELETE /identity/v3/projects/c1a839cda0114b38833e1be9e746515b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:35:02.650700 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a6082-ec0d-4fb2-9d01-6b321285aa13 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:02.650991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a6082-ec0d-4fb2-9d01-6b321285aa13 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:02.651273 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a6082-ec0d-4fb2-9d01-6b321285aa13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:02.670747 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-a28a6082-ec0d-4fb2-9d01-6b321285aa13 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:35:02.672016 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-a28a6082-ec0d-4fb2-9d01-6b321285aa13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:02.696319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:35:02.744695 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1425/2868] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:35:02.752490 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2e564-82f3-4d14-be6c-e5ff303a0cdc None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:02.752772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2e564-82f3-4d14-be6c-e5ff303a0cdc None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:02.753037 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2e564-82f3-4d14-be6c-e5ff303a0cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:02.772765 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-d4d2e564-82f3-4d14-be6c-e5ff303a0cdc None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:35:02.773950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-d4d2e564-82f3-4d14-be6c-e5ff303a0cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:02.781731 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1444/2869] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:35:02.785789 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a6082-ec0d-4fb2-9d01-6b321285aa13 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:02.795446 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a6082-ec0d-4fb2-9d01-6b321285aa13 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:02.800043 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2f36e-8049-4114-a140-6ab61a96cbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:02.800630 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2f36e-8049-4114-a140-6ab61a96cbf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:02.800929 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2f36e-8049-4114-a140-6ab61a96cbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:02.801189 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2f36e-8049-4114-a140-6ab61a96cbf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:02.801827 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2f36e-8049-4114-a140-6ab61a96cbf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:35:02.802131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2f36e-8049-4114-a140-6ab61a96cbf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:02.803729 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2f36e-8049-4114-a140-6ab61a96cbf5 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:02.815539 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1426/2870] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:35:02.820714 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2e564-82f3-4d14-be6c-e5ff303a0cdc None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:02.829664 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2e564-82f3-4d14-be6c-e5ff303a0cdc None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:02.834218 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be092860-7b9a-454b-928a-b92dd1092bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:02.834793 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be092860-7b9a-454b-928a-b92dd1092bfc admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:02.835065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be092860-7b9a-454b-928a-b92dd1092bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:02.835333 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be092860-7b9a-454b-928a-b92dd1092bfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:02.835990 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be092860-7b9a-454b-928a-b92dd1092bfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:02.837848 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be092860-7b9a-454b-928a-b92dd1092bfc admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:02.863461 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1445/2871] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:35:02 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:35:02.868231 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2f36e-8049-4114-a140-6ab61a96cbf5 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:02.878084 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2f36e-8049-4114-a140-6ab61a96cbf5 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:02.933389 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd7a5b-188b-4584-8d16-73ecb48eca64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:02.934021 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd7a5b-188b-4584-8d16-73ecb48eca64 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:02.934306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd7a5b-188b-4584-8d16-73ecb48eca64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:02.934580 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd7a5b-188b-4584-8d16-73ecb48eca64 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:02.935206 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd7a5b-188b-4584-8d16-73ecb48eca64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:02.937176 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd7a5b-188b-4584-8d16-73ecb48eca64 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:02.958707 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1427/2872] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:35:02 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:35:02.962859 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be092860-7b9a-454b-928a-b92dd1092bfc admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:02.973088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be092860-7b9a-454b-928a-b92dd1092bfc admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:02.977689 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f403b-86c0-4765-a8d9-eccbde908057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:02.978279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f403b-86c0-4765-a8d9-eccbde908057 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:02.978555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f403b-86c0-4765-a8d9-eccbde908057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:02.978825 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f403b-86c0-4765-a8d9-eccbde908057 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:02.979457 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f403b-86c0-4765-a8d9-eccbde908057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:02.981074 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f403b-86c0-4765-a8d9-eccbde908057 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:02.990755 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1446/2873] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:35:02.995026 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd7a5b-188b-4584-8d16-73ecb48eca64 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:03.004863 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd7a5b-188b-4584-8d16-73ecb48eca64 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:03.010051 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac69cf8-f267-4343-a70f-aba57d11fb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:03.010649 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac69cf8-f267-4343-a70f-aba57d11fb20 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:03.010931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac69cf8-f267-4343-a70f-aba57d11fb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:03.011211 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac69cf8-f267-4343-a70f-aba57d11fb20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:03.011885 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac69cf8-f267-4343-a70f-aba57d11fb20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:03.013480 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac69cf8-f267-4343-a70f-aba57d11fb20 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:03.023986 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1428/2874] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:35:03.028000 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f403b-86c0-4765-a8d9-eccbde908057 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:03.037972 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f403b-86c0-4765-a8d9-eccbde908057 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:03.042596 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016465bd-e91d-4794-8d37-e352b406c9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:03.043215 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016465bd-e91d-4794-8d37-e352b406c9d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:03.043486 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016465bd-e91d-4794-8d37-e352b406c9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:03.043752 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016465bd-e91d-4794-8d37-e352b406c9d7 admin admin] PATH_INFO: `/v3/projects/05b8bf0958064b6cac375e4b7847fc2d/users/59190c22ec6144d89218c52d8d18fba6/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:03.058219 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016465bd-e91d-4794-8d37-e352b406c9d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b8bf0958064b6cac375e4b7847fc2d, user_id=59190c22ec6144d89218c52d8d18fba6, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:03.060876 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016465bd-e91d-4794-8d37-e352b406c9d7 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:03.074084 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1447/2875] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:35:03 2022] PUT /identity/v3/projects/05b8bf0958064b6cac375e4b7847fc2d/users/59190c22ec6144d89218c52d8d18fba6/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:35:03.085966 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbd59b-bdd5-4304-ade5-ad4c37725f71 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:03.086333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbd59b-bdd5-4304-ade5-ad4c37725f71 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:03.086612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbd59b-bdd5-4304-ade5-ad4c37725f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:03.115086 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-40dbd59b-bdd5-4304-ade5-ad4c37725f71 None None] MFA Rules not processed for user `59190c22ec6144d89218c52d8d18fba6`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:35:03.116319 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-40dbd59b-bdd5-4304-ade5-ad4c37725f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:03.178856 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1429/2876] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:35:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:35:03.189869 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016465bd-e91d-4794-8d37-e352b406c9d7 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:03.191628 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-016465bd-e91d-4794-8d37-e352b406c9d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:03.213199 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:35:03.251469 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016465bd-e91d-4794-8d37-e352b406c9d7 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:03.255076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8286c3-7703-4df2-afcc-adf264f8f5a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:03.255671 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8286c3-7703-4df2-afcc-adf264f8f5a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:03.255986 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8286c3-7703-4df2-afcc-adf264f8f5a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:03.256258 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8286c3-7703-4df2-afcc-adf264f8f5a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:03.264269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8286c3-7703-4df2-afcc-adf264f8f5a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:03.265673 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8286c3-7703-4df2-afcc-adf264f8f5a5 service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:03.274303 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1448/2877] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:35:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:35:13.069315 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbd59b-bdd5-4304-ade5-ad4c37725f71 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:13.089140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:35:13.134268 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbd59b-bdd5-4304-ade5-ad4c37725f71 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:13.138693 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3042f1-81af-4b5a-80f2-211971390db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:13.139337 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3042f1-81af-4b5a-80f2-211971390db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:13.139612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3042f1-81af-4b5a-80f2-211971390db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:13.139923 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3042f1-81af-4b5a-80f2-211971390db3 admin admin] PATH_INFO: `/v3/users/59190c22ec6144d89218c52d8d18fba6` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:13.141020 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3042f1-81af-4b5a-80f2-211971390db3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59190c22ec6144d89218c52d8d18fba6)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:13.142920 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3042f1-81af-4b5a-80f2-211971390db3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:13.191376 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4c3042f1-81af-4b5a-80f2-211971390db3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59190c22ec6144d89218c52d8d18fba6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:35:13.198931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4c3042f1-81af-4b5a-80f2-211971390db3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59190c22ec6144d89218c52d8d18fba6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:35:13.199548 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4c3042f1-81af-4b5a-80f2-211971390db3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59190c22ec6144d89218c52d8d18fba6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:35:13.200201 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4c3042f1-81af-4b5a-80f2-211971390db3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59190c22ec6144d89218c52d8d18fba6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:35:13.212429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-4c3042f1-81af-4b5a-80f2-211971390db3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59190c22ec6144d89218c52d8d18fba6'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:35:13.224415 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1430/2878] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:35:13 2022] DELETE /identity/v3/users/59190c22ec6144d89218c52d8d18fba6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:35:13.229269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105ba9f-a03b-4e83-bfa4-bba6588e12fb None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:13.229797 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105ba9f-a03b-4e83-bfa4-bba6588e12fb None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:13.229797 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105ba9f-a03b-4e83-bfa4-bba6588e12fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:13.248361 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-3105ba9f-a03b-4e83-bfa4-bba6588e12fb None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:35:13.249543 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3105ba9f-a03b-4e83-bfa4-bba6588e12fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:13.305627 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1449/2879] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:35:13.315132 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3042f1-81af-4b5a-80f2-211971390db3 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:13.316530 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4c3042f1-81af-4b5a-80f2-211971390db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:13.373268 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3042f1-81af-4b5a-80f2-211971390db3 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:13.377010 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb63452-edfc-4673-97cd-9ae64b498d0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:13.377614 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb63452-edfc-4673-97cd-9ae64b498d0d service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:13.377894 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb63452-edfc-4673-97cd-9ae64b498d0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:13.378165 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb63452-edfc-4673-97cd-9ae64b498d0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:13.387250 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb63452-edfc-4673-97cd-9ae64b498d0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:13.388714 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb63452-edfc-4673-97cd-9ae64b498d0d service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:13.398041 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1431/2880] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:35:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:35:13.663340 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105ba9f-a03b-4e83-bfa4-bba6588e12fb None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:13.664780 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3105ba9f-a03b-4e83-bfa4-bba6588e12fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:13.681352 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105ba9f-a03b-4e83-bfa4-bba6588e12fb None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:13.686330 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a008cc-9785-4c34-abf6-05d4a31ca342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:13.686843 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a008cc-9785-4c34-abf6-05d4a31ca342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:13.687122 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a008cc-9785-4c34-abf6-05d4a31ca342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:13.687395 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a008cc-9785-4c34-abf6-05d4a31ca342 admin admin] PATH_INFO: `/v3/projects/05b8bf0958064b6cac375e4b7847fc2d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:13.688705 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a008cc-9785-4c34-abf6-05d4a31ca342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05b8bf0958064b6cac375e4b7847fc2d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:13.690631 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a008cc-9785-4c34-abf6-05d4a31ca342 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:13.737843 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c8a008cc-9785-4c34-abf6-05d4a31ca342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05b8bf0958064b6cac375e4b7847fc2d', 'request_id': 'req-c8a008cc-9785-4c34-abf6-05d4a31ca342'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:35:13.744088 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c8a008cc-9785-4c34-abf6-05d4a31ca342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05b8bf0958064b6cac375e4b7847fc2d', 'request_id': 'req-c8a008cc-9785-4c34-abf6-05d4a31ca342'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:35:13.749624 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c8a008cc-9785-4c34-abf6-05d4a31ca342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05b8bf0958064b6cac375e4b7847fc2d', 'request_id': 'req-c8a008cc-9785-4c34-abf6-05d4a31ca342'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:35:13.762641 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c8a008cc-9785-4c34-abf6-05d4a31ca342 admin admin] The token cache is being invalidate because project 05b8bf0958064b6cac375e4b7847fc2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:35:13.762932 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c8a008cc-9785-4c34-abf6-05d4a31ca342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:35:13.763578 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c8a008cc-9785-4c34-abf6-05d4a31ca342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:35:13.765076 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1450/2881] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:35:13 2022] DELETE /identity/v3/projects/05b8bf0958064b6cac375e4b7847fc2d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:35:13.769883 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc27fc7-191a-4fa6-9207-431199beac84 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:13.770171 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc27fc7-191a-4fa6-9207-431199beac84 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:13.770446 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc27fc7-191a-4fa6-9207-431199beac84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:13.790612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8dc27fc7-191a-4fa6-9207-431199beac84 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:35:13.792029 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8dc27fc7-191a-4fa6-9207-431199beac84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:13.835978 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:35:13.854728 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1432/2882] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:35:13.862466 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e32f02-6df7-48a0-b73d-c2f3f49c3c51 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:13.862772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e32f02-6df7-48a0-b73d-c2f3f49c3c51 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:13.863045 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e32f02-6df7-48a0-b73d-c2f3f49c3c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:13.882399 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-a1e32f02-6df7-48a0-b73d-c2f3f49c3c51 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:35:13.884266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a1e32f02-6df7-48a0-b73d-c2f3f49c3c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:13.892147 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1451/2883] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:35:13.896284 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc27fc7-191a-4fa6-9207-431199beac84 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:13.906448 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc27fc7-191a-4fa6-9207-431199beac84 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:13.910976 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8eb9af-9f11-496a-ac79-58b9fc351c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:13.911558 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8eb9af-9f11-496a-ac79-58b9fc351c89 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:13.911874 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8eb9af-9f11-496a-ac79-58b9fc351c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:13.912148 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8eb9af-9f11-496a-ac79-58b9fc351c89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:13.912783 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8eb9af-9f11-496a-ac79-58b9fc351c89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:35:13.913081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8eb9af-9f11-496a-ac79-58b9fc351c89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:13.914663 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8eb9af-9f11-496a-ac79-58b9fc351c89 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:13.926013 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1433/2884] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:35:13.930232 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e32f02-6df7-48a0-b73d-c2f3f49c3c51 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:13.939279 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e32f02-6df7-48a0-b73d-c2f3f49c3c51 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:13.944185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d1e15-df6d-4844-b178-57cc7888359d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:13.944769 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d1e15-df6d-4844-b178-57cc7888359d admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:13.945563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d1e15-df6d-4844-b178-57cc7888359d admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:13.945834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d1e15-df6d-4844-b178-57cc7888359d admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:13.946466 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d1e15-df6d-4844-b178-57cc7888359d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:13.948365 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d1e15-df6d-4844-b178-57cc7888359d admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:13.975240 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1452/2885] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:35:13 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:35:13.979272 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8eb9af-9f11-496a-ac79-58b9fc351c89 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:13.989306 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8eb9af-9f11-496a-ac79-58b9fc351c89 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:14.053429 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a9e68-d4ea-4a71-a2ce-73af76e96717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:14.054028 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a9e68-d4ea-4a71-a2ce-73af76e96717 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:14.054919 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a9e68-d4ea-4a71-a2ce-73af76e96717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:14.055090 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a9e68-d4ea-4a71-a2ce-73af76e96717 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:14.055705 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a9e68-d4ea-4a71-a2ce-73af76e96717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:14.057653 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a9e68-d4ea-4a71-a2ce-73af76e96717 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:14.080101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1434/2886] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:35:13 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:35:14.084405 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d1e15-df6d-4844-b178-57cc7888359d admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:14.094831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d1e15-df6d-4844-b178-57cc7888359d admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:14.099444 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d51f0-6f1c-4555-ac38-5e0b51bb4613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:14.100060 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d51f0-6f1c-4555-ac38-5e0b51bb4613 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:14.100342 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d51f0-6f1c-4555-ac38-5e0b51bb4613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:14.100616 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d51f0-6f1c-4555-ac38-5e0b51bb4613 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:14.101312 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d51f0-6f1c-4555-ac38-5e0b51bb4613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:14.102913 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d51f0-6f1c-4555-ac38-5e0b51bb4613 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:14.113360 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1453/2887] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:35:14.117321 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a9e68-d4ea-4a71-a2ce-73af76e96717 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:14.127511 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a9e68-d4ea-4a71-a2ce-73af76e96717 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:14.132297 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6abeb-2186-476f-a442-93d21588b50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:14.133432 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6abeb-2186-476f-a442-93d21588b50b admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:14.133715 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6abeb-2186-476f-a442-93d21588b50b admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:14.133991 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6abeb-2186-476f-a442-93d21588b50b admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:14.134634 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6abeb-2186-476f-a442-93d21588b50b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:14.136270 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6abeb-2186-476f-a442-93d21588b50b admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:14.146775 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1435/2888] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:35:14.150883 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d51f0-6f1c-4555-ac38-5e0b51bb4613 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:14.160710 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d51f0-6f1c-4555-ac38-5e0b51bb4613 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:14.165290 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb23cfc-74ad-4cf0-8f80-b88f23d29215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:14.165925 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb23cfc-74ad-4cf0-8f80-b88f23d29215 admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:14.166204 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb23cfc-74ad-4cf0-8f80-b88f23d29215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:14.166480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb23cfc-74ad-4cf0-8f80-b88f23d29215 admin admin] PATH_INFO: `/v3/projects/1b4bae2bd5f34bdc88c81e24590ade57/users/26a9a15e88a648bb93cfc5c49bd17786/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:14.168684 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:35:14.184887 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb23cfc-74ad-4cf0-8f80-b88f23d29215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b4bae2bd5f34bdc88c81e24590ade57, user_id=26a9a15e88a648bb93cfc5c49bd17786, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:14.187514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb23cfc-74ad-4cf0-8f80-b88f23d29215 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:14.201834 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1454/2889] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:35:14 2022] PUT /identity/v3/projects/1b4bae2bd5f34bdc88c81e24590ade57/users/26a9a15e88a648bb93cfc5c49bd17786/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:35:14.206787 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0958-3d57-4324-9934-ae1e44786cfd None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:14.207077 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0958-3d57-4324-9934-ae1e44786cfd None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:14.207357 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0958-3d57-4324-9934-ae1e44786cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:14.226110 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-f0aa0958-3d57-4324-9934-ae1e44786cfd None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:35:14.227374 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f0aa0958-3d57-4324-9934-ae1e44786cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:14.287052 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1436/2890] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:35:14.296737 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb23cfc-74ad-4cf0-8f80-b88f23d29215 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:14.298425 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-4bb23cfc-74ad-4cf0-8f80-b88f23d29215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:14.355922 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb23cfc-74ad-4cf0-8f80-b88f23d29215 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:14.359563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c0a47-6024-4239-a652-93291fec7185 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:14.360200 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c0a47-6024-4239-a652-93291fec7185 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:14.360479 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c0a47-6024-4239-a652-93291fec7185 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:14.360752 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c0a47-6024-4239-a652-93291fec7185 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:14.370189 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c0a47-6024-4239-a652-93291fec7185 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:14.371550 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c0a47-6024-4239-a652-93291fec7185 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:14.380921 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1455/2891] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:35:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 11:35:14.971628 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0b916-8780-4ee3-ba30-83ace81ed771 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:14.972018 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0b916-8780-4ee3-ba30-83ace81ed771 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:14.972359 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0b916-8780-4ee3-ba30-83ace81ed771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:15.000410 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-8da0b916-8780-4ee3-ba30-83ace81ed771 None None] MFA Rules not processed for user `26a9a15e88a648bb93cfc5c49bd17786`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:35:15.001640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-8da0b916-8780-4ee3-ba30-83ace81ed771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:15.035294 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:35:15.069806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1437/2892] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:35:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:35:15.082347 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c0a47-6024-4239-a652-93291fec7185 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:15.083740 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3a0c0a47-6024-4239-a652-93291fec7185 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:15.141717 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c0a47-6024-4239-a652-93291fec7185 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:15.145464 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803740e-6537-4d60-8157-05c760ca29cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9f494df37254102b339df35a151e075', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:15.146062 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803740e-6537-4d60-8157-05c760ca29cb service cinder] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:15.146928 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803740e-6537-4d60-8157-05c760ca29cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:15.147214 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803740e-6537-4d60-8157-05c760ca29cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:15.156010 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803740e-6537-4d60-8157-05c760ca29cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:15.157433 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803740e-6537-4d60-8157-05c760ca29cb service cinder] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:15.166936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1456/2893] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:35:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:35:16.521535 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87af7e7-48c2-4eae-a133-314c1dfe40ef None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:16.522101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87af7e7-48c2-4eae-a133-314c1dfe40ef None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:16.522101 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87af7e7-48c2-4eae-a133-314c1dfe40ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:16.542370 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-e87af7e7-48c2-4eae-a133-314c1dfe40ef None None] MFA Rules not processed for user `26a9a15e88a648bb93cfc5c49bd17786`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:35:16.543543 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e87af7e7-48c2-4eae-a133-314c1dfe40ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:16.550431 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1438/2894] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:35:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:35:16.784698 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803740e-6537-4d60-8157-05c760ca29cb service cinder] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:16.794465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803740e-6537-4d60-8157-05c760ca29cb service cinder] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:16.798057 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aefc64-5eeb-457e-ad8f-9ae80331c0a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:16.798641 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aefc64-5eeb-457e-ad8f-9ae80331c0a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:16.798916 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aefc64-5eeb-457e-ad8f-9ae80331c0a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:16.799185 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aefc64-5eeb-457e-ad8f-9ae80331c0a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:16.800969 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e0aefc64-5eeb-457e-ad8f-9ae80331c0a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:16.810372 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aefc64-5eeb-457e-ad8f-9ae80331c0a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:16.811794 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aefc64-5eeb-457e-ad8f-9ae80331c0a3 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:16.820355 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1457/2895] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:35:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:35:17.151582 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87af7e7-48c2-4eae-a133-314c1dfe40ef None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:17.153333 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e87af7e7-48c2-4eae-a133-314c1dfe40ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:17.228066 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87af7e7-48c2-4eae-a133-314c1dfe40ef None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:17.232995 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05178325-9242-4abc-a856-44e79f940159 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e7a18dc4a4d4d35a05f1aafbaaf9c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:17.233590 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05178325-9242-4abc-a856-44e79f940159 service placement] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:17.233872 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05178325-9242-4abc-a856-44e79f940159 service placement] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:17.234145 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05178325-9242-4abc-a856-44e79f940159 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:17.235930 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-05178325-9242-4abc-a856-44e79f940159 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:17.248818 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05178325-9242-4abc-a856-44e79f940159 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:17.250442 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05178325-9242-4abc-a856-44e79f940159 service placement] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:17.261316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1439/2896] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:35:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:35:19.641828 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aefc64-5eeb-457e-ad8f-9ae80331c0a3 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:19.643899 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-e0aefc64-5eeb-457e-ad8f-9ae80331c0a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:19.738147 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aefc64-5eeb-457e-ad8f-9ae80331c0a3 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:19.744276 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea89dc-b18a-45c5-955d-ff3720a0d835 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:19.744889 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea89dc-b18a-45c5-955d-ff3720a0d835 service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:19.745168 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea89dc-b18a-45c5-955d-ff3720a0d835 service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:19.745441 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea89dc-b18a-45c5-955d-ff3720a0d835 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:19.747208 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-a7ea89dc-b18a-45c5-955d-ff3720a0d835 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:19.760543 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea89dc-b18a-45c5-955d-ff3720a0d835 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:19.762271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea89dc-b18a-45c5-955d-ff3720a0d835 service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:19.768751 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:35:19.775011 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1458/2897] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:35:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:35:25.494577 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05178325-9242-4abc-a856-44e79f940159 service placement] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:35:25.508524 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05178325-9242-4abc-a856-44e79f940159 service placement] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:35:25.513931 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5ce55-8ffc-4d36-b3a9-bf6784c1ad98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:35:25.514525 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5ce55-8ffc-4d36-b3a9-bf6784c1ad98 service nova] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:35:25.514800 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5ce55-8ffc-4d36-b3a9-bf6784c1ad98 service nova] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:35:25.515070 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5ce55-8ffc-4d36-b3a9-bf6784c1ad98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:35:25.516904 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-caa5ce55-8ffc-4d36-b3a9-bf6784c1ad98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:35:25.529081 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5ce55-8ffc-4d36-b3a9-bf6784c1ad98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:35:25.530645 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5ce55-8ffc-4d36-b3a9-bf6784c1ad98 service nova] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:35:25.541497 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1440/2898] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:35:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:35:35.536599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=52 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:36:14.346116 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6dda2-8af7-4312-a784-fe9817b13a82 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:36:14.346645 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6dda2-8af7-4312-a784-fe9817b13a82 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:36:14.346645 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6dda2-8af7-4312-a784-fe9817b13a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:36:14.365331 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-35f6dda2-8af7-4312-a784-fe9817b13a82 None None] MFA Rules not processed for user `26a9a15e88a648bb93cfc5c49bd17786`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:36:14.366535 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-35f6dda2-8af7-4312-a784-fe9817b13a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:36:14.373355 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1459/2899] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:36:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:36:14.602981 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5ce55-8ffc-4d36-b3a9-bf6784c1ad98 service nova] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:36:14.612399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5ce55-8ffc-4d36-b3a9-bf6784c1ad98 service nova] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:36:14.615998 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed6c28-09e7-4f07-8576-5d29abfb9cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:36:14.616589 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed6c28-09e7-4f07-8576-5d29abfb9cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:36:14.616870 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed6c28-09e7-4f07-8576-5d29abfb9cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:36:14.617142 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed6c28-09e7-4f07-8576-5d29abfb9cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:36:14.618837 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-79ed6c28-09e7-4f07-8576-5d29abfb9cf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:36:14.628822 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed6c28-09e7-4f07-8576-5d29abfb9cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:36:14.630234 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed6c28-09e7-4f07-8576-5d29abfb9cf2 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:36:14.638729 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1441/2900] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:36:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:36:24.359932 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:36:24.634484 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=6 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:36:42.119230 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6dda2-8af7-4312-a784-fe9817b13a82 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:36:42.131034 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6dda2-8af7-4312-a784-fe9817b13a82 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:36:42.136709 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17279cb7-6fd9-4111-883a-130ad0444d08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e7a18dc4a4d4d35a05f1aafbaaf9c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:36:42.137327 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17279cb7-6fd9-4111-883a-130ad0444d08 service placement] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:36:42.137601 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17279cb7-6fd9-4111-883a-130ad0444d08 service placement] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:36:42.137872 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17279cb7-6fd9-4111-883a-130ad0444d08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:36:42.139621 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-17279cb7-6fd9-4111-883a-130ad0444d08 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:36:42.151648 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17279cb7-6fd9-4111-883a-130ad0444d08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:36:42.153226 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17279cb7-6fd9-4111-883a-130ad0444d08 service placement] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:36:42.162898 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1460/2901] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:36:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:36:52.158814 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=6 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:37:07.612406 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed6c28-09e7-4f07-8576-5d29abfb9cf2 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:07.624174 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed6c28-09e7-4f07-8576-5d29abfb9cf2 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:07.628657 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3c4c4-fa9b-44e4-bd83-bac3116527a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:07.629334 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3c4c4-fa9b-44e4-bd83-bac3116527a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:07.629726 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3c4c4-fa9b-44e4-bd83-bac3116527a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:07.630001 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3c4c4-fa9b-44e4-bd83-bac3116527a3 admin admin] PATH_INFO: `/v3/users/26a9a15e88a648bb93cfc5c49bd17786` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:07.631283 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3c4c4-fa9b-44e4-bd83-bac3116527a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26a9a15e88a648bb93cfc5c49bd17786)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:07.633662 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3c4c4-fa9b-44e4-bd83-bac3116527a3 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:07.685604 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f8f3c4c4-fa9b-44e4-bd83-bac3116527a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26a9a15e88a648bb93cfc5c49bd17786'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:37:07.695394 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f8f3c4c4-fa9b-44e4-bd83-bac3116527a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26a9a15e88a648bb93cfc5c49bd17786'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:37:07.696151 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f8f3c4c4-fa9b-44e4-bd83-bac3116527a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26a9a15e88a648bb93cfc5c49bd17786'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:37:07.696760 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f8f3c4c4-fa9b-44e4-bd83-bac3116527a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26a9a15e88a648bb93cfc5c49bd17786'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:37:07.709643 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-f8f3c4c4-fa9b-44e4-bd83-bac3116527a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26a9a15e88a648bb93cfc5c49bd17786'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:37:07.722244 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1442/2902] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:37:07 2022] DELETE /identity/v3/users/26a9a15e88a648bb93cfc5c49bd17786 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:37:07.901148 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17279cb7-6fd9-4111-883a-130ad0444d08 service placement] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:07.911887 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17279cb7-6fd9-4111-883a-130ad0444d08 service placement] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:07.968908 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831248bc-8bef-4810-a5b2-72e794fcca46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:07.969521 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831248bc-8bef-4810-a5b2-72e794fcca46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:07.969799 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831248bc-8bef-4810-a5b2-72e794fcca46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:07.970070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831248bc-8bef-4810-a5b2-72e794fcca46 admin admin] PATH_INFO: `/v3/projects/1b4bae2bd5f34bdc88c81e24590ade57` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:07.971178 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831248bc-8bef-4810-a5b2-72e794fcca46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b4bae2bd5f34bdc88c81e24590ade57)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:07.973083 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831248bc-8bef-4810-a5b2-72e794fcca46 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:08.018318 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-831248bc-8bef-4810-a5b2-72e794fcca46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b4bae2bd5f34bdc88c81e24590ade57', 'request_id': 'req-831248bc-8bef-4810-a5b2-72e794fcca46'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:37:08.024847 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-831248bc-8bef-4810-a5b2-72e794fcca46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b4bae2bd5f34bdc88c81e24590ade57', 'request_id': 'req-831248bc-8bef-4810-a5b2-72e794fcca46'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:37:08.030307 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-831248bc-8bef-4810-a5b2-72e794fcca46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b4bae2bd5f34bdc88c81e24590ade57', 'request_id': 'req-831248bc-8bef-4810-a5b2-72e794fcca46'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:37:08.042661 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-831248bc-8bef-4810-a5b2-72e794fcca46 admin admin] The token cache is being invalidate because project 1b4bae2bd5f34bdc88c81e24590ade57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81461) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:37:08.042953 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-831248bc-8bef-4810-a5b2-72e794fcca46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:37:08.043575 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-831248bc-8bef-4810-a5b2-72e794fcca46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:37:08.045096 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1461/2903] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:37:07 2022] DELETE /identity/v3/projects/1b4bae2bd5f34bdc88c81e24590ade57 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:37:08.050016 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f6c03-6e89-40f7-8a6b-bd4ea4728b83 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:08.050433 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f6c03-6e89-40f7-8a6b-bd4ea4728b83 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:08.050714 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f6c03-6e89-40f7-8a6b-bd4ea4728b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:08.073213 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-2e7f6c03-6e89-40f7-8a6b-bd4ea4728b83 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:37:08.074435 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-2e7f6c03-6e89-40f7-8a6b-bd4ea4728b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:37:08.132544 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1443/2904] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:37:08.140772 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5d0e8-74d8-422a-b7df-43a56f6480c6 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:08.141059 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5d0e8-74d8-422a-b7df-43a56f6480c6 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:08.141333 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5d0e8-74d8-422a-b7df-43a56f6480c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:08.161210 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-26e5d0e8-74d8-422a-b7df-43a56f6480c6 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:37:08.162418 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-26e5d0e8-74d8-422a-b7df-43a56f6480c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:37:08.170783 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1462/2905] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:37:08.174971 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f6c03-6e89-40f7-8a6b-bd4ea4728b83 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:08.184834 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f6c03-6e89-40f7-8a6b-bd4ea4728b83 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:08.189670 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8afacb-6360-44b6-9612-d8041f91fca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:08.190260 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8afacb-6360-44b6-9612-d8041f91fca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:08.190534 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8afacb-6360-44b6-9612-d8041f91fca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:08.190801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8afacb-6360-44b6-9612-d8041f91fca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:08.191442 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8afacb-6360-44b6-9612-d8041f91fca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 11:37:08.191794 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8afacb-6360-44b6-9612-d8041f91fca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:08.193380 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8afacb-6360-44b6-9612-d8041f91fca7 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:08.205189 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1444/2906] 10.222.0.81 () {58 vars in 1153 bytes} [Mon Sep 19 11:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 11:37:08.209332 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5d0e8-74d8-422a-b7df-43a56f6480c6 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:08.219555 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5d0e8-74d8-422a-b7df-43a56f6480c6 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:08.224640 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6d215-3839-44ae-a106-ef731d281bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:08.225236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6d215-3839-44ae-a106-ef731d281bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:08.225520 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6d215-3839-44ae-a106-ef731d281bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:08.225854 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6d215-3839-44ae-a106-ef731d281bb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:08.226553 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6d215-3839-44ae-a106-ef731d281bb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:08.228490 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6d215-3839-44ae-a106-ef731d281bb7 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:08.259280 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1463/2907] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:37:08 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:37:08.260462 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8afacb-6360-44b6-9612-d8041f91fca7 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:08.270374 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8afacb-6360-44b6-9612-d8041f91fca7 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:08.327082 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb0bd5-a306-49d5-b2a8-5277fd923c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:08.327667 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb0bd5-a306-49d5-b2a8-5277fd923c6a admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:08.327970 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb0bd5-a306-49d5-b2a8-5277fd923c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:08.328239 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb0bd5-a306-49d5-b2a8-5277fd923c6a admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:08.328851 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb0bd5-a306-49d5-b2a8-5277fd923c6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:08.330747 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb0bd5-a306-49d5-b2a8-5277fd923c6a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:08.339124 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:37:08.353645 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1445/2908] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:37:08 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:37:08.357950 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6d215-3839-44ae-a106-ef731d281bb7 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:08.368291 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6d215-3839-44ae-a106-ef731d281bb7 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:08.372925 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0ab11-10db-466b-9731-2d4d224cbbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:08.373515 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0ab11-10db-466b-9731-2d4d224cbbd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:08.373891 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0ab11-10db-466b-9731-2d4d224cbbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:08.374161 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0ab11-10db-466b-9731-2d4d224cbbd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:08.374820 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0ab11-10db-466b-9731-2d4d224cbbd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:08.376450 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0ab11-10db-466b-9731-2d4d224cbbd0 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:08.386830 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1464/2909] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:37:08.390602 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb0bd5-a306-49d5-b2a8-5277fd923c6a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:08.401445 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb0bd5-a306-49d5-b2a8-5277fd923c6a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:08.406186 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bf43f-b8fc-488b-afc7-0d43cc5c939d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:08.406775 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bf43f-b8fc-488b-afc7-0d43cc5c939d admin admin] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:08.407057 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bf43f-b8fc-488b-afc7-0d43cc5c939d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:08.407328 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bf43f-b8fc-488b-afc7-0d43cc5c939d admin admin] PATH_INFO: `/v3/roles` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:08.407992 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bf43f-b8fc-488b-afc7-0d43cc5c939d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:08.409593 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bf43f-b8fc-488b-afc7-0d43cc5c939d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:08.420595 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1446/2910] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:37:08.425325 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0ab11-10db-466b-9731-2d4d224cbbd0 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:08.436090 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0ab11-10db-466b-9731-2d4d224cbbd0 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:08.440982 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e49fb-37e7-42a5-a7de-f43ecc67c43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:08.441621 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e49fb-37e7-42a5-a7de-f43ecc67c43f admin admin] REQUEST_METHOD: `PUT` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:08.441901 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e49fb-37e7-42a5-a7de-f43ecc67c43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:08.442177 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e49fb-37e7-42a5-a7de-f43ecc67c43f admin admin] PATH_INFO: `/v3/projects/67f0a300a82b480d8f4a2c0a19f94fb6/users/1d4fe64743d747ccadb431ce88fbadcb/roles/0c30c75e227c416da6e4814bce700e8d` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:08.457002 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e49fb-37e7-42a5-a7de-f43ecc67c43f admin admin] RBAC: Authorizing `identity:create_grant(project_id=67f0a300a82b480d8f4a2c0a19f94fb6, user_id=1d4fe64743d747ccadb431ce88fbadcb, role_id=0c30c75e227c416da6e4814bce700e8d)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:08.459608 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e49fb-37e7-42a5-a7de-f43ecc67c43f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:08.472017 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1465/2911] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:37:08 2022] PUT /identity/v3/projects/67f0a300a82b480d8f4a2c0a19f94fb6/users/1d4fe64743d747ccadb431ce88fbadcb/roles/0c30c75e227c416da6e4814bce700e8d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:37:08.477525 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d3b91-0d57-4b48-8c65-9e9dde712428 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:08.477811 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d3b91-0d57-4b48-8c65-9e9dde712428 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:08.478110 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d3b91-0d57-4b48-8c65-9e9dde712428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:08.497336 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-3d6d3b91-0d57-4b48-8c65-9e9dde712428 None None] MFA Rules not processed for user `dd316848081f4ae2ab8647785189b3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:37:08.498563 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-3d6d3b91-0d57-4b48-8c65-9e9dde712428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:37:08.560087 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1447/2912] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:37:08.570510 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e49fb-37e7-42a5-a7de-f43ecc67c43f admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:08.572253 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-3b7e49fb-37e7-42a5-a7de-f43ecc67c43f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:37:08.602602 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:37:08.630618 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e49fb-37e7-42a5-a7de-f43ecc67c43f admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:08.634886 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998610dc-1c51-4048-ae3c-dc2f59f69884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:08.635484 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998610dc-1c51-4048-ae3c-dc2f59f69884 service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:08.635788 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998610dc-1c51-4048-ae3c-dc2f59f69884 service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:08.636065 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998610dc-1c51-4048-ae3c-dc2f59f69884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:08.645073 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998610dc-1c51-4048-ae3c-dc2f59f69884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:08.646421 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998610dc-1c51-4048-ae3c-dc2f59f69884 service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:08.655866 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1466/2913] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:37:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:37:12.612489 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b85bd5-c130-4f95-9c6c-d0b2b3951797 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:12.613058 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b85bd5-c130-4f95-9c6c-d0b2b3951797 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:12.613058 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b85bd5-c130-4f95-9c6c-d0b2b3951797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:12.641856 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-60b85bd5-c130-4f95-9c6c-d0b2b3951797 None None] MFA Rules not processed for user `1d4fe64743d747ccadb431ce88fbadcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:37:12.643097 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-60b85bd5-c130-4f95-9c6c-d0b2b3951797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:37:12.714414 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1448/2914] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:37:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:37:12.718935 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998610dc-1c51-4048-ae3c-dc2f59f69884 service neutron] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:12.729062 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998610dc-1c51-4048-ae3c-dc2f59f69884 service neutron] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:12.733681 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2de47-0c2d-446b-9386-5fa28a496313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:12.734286 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2de47-0c2d-446b-9386-5fa28a496313 admin admin] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:12.734561 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2de47-0c2d-446b-9386-5fa28a496313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:12.734831 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2de47-0c2d-446b-9386-5fa28a496313 admin admin] PATH_INFO: `/v3/projects` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:12.735474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b2de47-0c2d-446b-9386-5fa28a496313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:12.737416 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b2de47-0c2d-446b-9386-5fa28a496313 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:12.768266 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1467/2915] 10.222.0.81 () {60 vars in 1153 bytes} [Mon Sep 19 11:37:12 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 11:37:12.769383 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b85bd5-c130-4f95-9c6c-d0b2b3951797 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:12.781229 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b85bd5-c130-4f95-9c6c-d0b2b3951797 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:12.838826 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d0ce8-fd8d-40d4-8f54-62632d540604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:12.839455 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d0ce8-fd8d-40d4-8f54-62632d540604 admin admin] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:12.839734 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d0ce8-fd8d-40d4-8f54-62632d540604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:12.840053 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d0ce8-fd8d-40d4-8f54-62632d540604 admin admin] PATH_INFO: `/v3/users` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:12.840704 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d0ce8-fd8d-40d4-8f54-62632d540604 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:12.842672 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d0ce8-fd8d-40d4-8f54-62632d540604 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:12.859159 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:37:12.866806 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1449/2916] 10.222.0.81 () {60 vars in 1144 bytes} [Mon Sep 19 11:37:12 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 11:37:12.870821 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2de47-0c2d-446b-9386-5fa28a496313 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:12.882155 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2de47-0c2d-446b-9386-5fa28a496313 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:12.887070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943acec-6fa1-4cf1-8d36-f3ae953cf988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:12.887672 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943acec-6fa1-4cf1-8d36-f3ae953cf988 admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:12.888038 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943acec-6fa1-4cf1-8d36-f3ae953cf988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:12.888310 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943acec-6fa1-4cf1-8d36-f3ae953cf988 admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:12.888948 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943acec-6fa1-4cf1-8d36-f3ae953cf988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:12.890621 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943acec-6fa1-4cf1-8d36-f3ae953cf988 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:12.902028 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1468/2917] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 11:37:12.906243 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d0ce8-fd8d-40d4-8f54-62632d540604 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:12.916511 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d0ce8-fd8d-40d4-8f54-62632d540604 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:12.921323 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d31eeb-e4d3-4060-9613-fe724814b46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:12.921984 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d31eeb-e4d3-4060-9613-fe724814b46d admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:12.922267 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d31eeb-e4d3-4060-9613-fe724814b46d admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:12.922543 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d31eeb-e4d3-4060-9613-fe724814b46d admin admin] PATH_INFO: `/v3/projects/742245b438694e51aab9349ad8364995/users/a0705f7ef17545b8aa15f3d167cae2e1/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:12.937297 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d31eeb-e4d3-4060-9613-fe724814b46d admin admin] RBAC: Authorizing `identity:create_grant(project_id=742245b438694e51aab9349ad8364995, user_id=a0705f7ef17545b8aa15f3d167cae2e1, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:12.940242 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d31eeb-e4d3-4060-9613-fe724814b46d admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:12.953236 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1450/2918] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:37:12 2022] PUT /identity/v3/projects/742245b438694e51aab9349ad8364995/users/a0705f7ef17545b8aa15f3d167cae2e1/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:37:12.957168 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943acec-6fa1-4cf1-8d36-f3ae953cf988 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:12.967856 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943acec-6fa1-4cf1-8d36-f3ae953cf988 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:13.024386 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eba645-3145-4e5e-822b-0774f88fea3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:13.024989 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eba645-3145-4e5e-822b-0774f88fea3f admin admin] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:13.025269 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eba645-3145-4e5e-822b-0774f88fea3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:13.025543 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eba645-3145-4e5e-822b-0774f88fea3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:13.026172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eba645-3145-4e5e-822b-0774f88fea3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:13.027747 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eba645-3145-4e5e-822b-0774f88fea3f admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:13.037275 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1469/2919] 10.222.0.81 () {58 vars in 1122 bytes} [Mon Sep 19 11:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 11:37:13.041534 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d31eeb-e4d3-4060-9613-fe724814b46d admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:13.051606 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d31eeb-e4d3-4060-9613-fe724814b46d admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:13.056680 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c16e5a-76bc-4a90-950c-a38cbdac3077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:13.057316 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c16e5a-76bc-4a90-950c-a38cbdac3077 admin admin] REQUEST_METHOD: `PUT` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:13.057591 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c16e5a-76bc-4a90-950c-a38cbdac3077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:13.057864 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c16e5a-76bc-4a90-950c-a38cbdac3077 admin admin] PATH_INFO: `/v3/projects/742245b438694e51aab9349ad8364995/users/a0705f7ef17545b8aa15f3d167cae2e1/roles/753f8b35d5f0458d8f62d0ad6e28033f` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:13.059813 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c16e5a-76bc-4a90-950c-a38cbdac3077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=742245b438694e51aab9349ad8364995, user_id=a0705f7ef17545b8aa15f3d167cae2e1, role_id=753f8b35d5f0458d8f62d0ad6e28033f)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:13.062478 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c16e5a-76bc-4a90-950c-a38cbdac3077 admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:13.074181 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1451/2920] 10.222.0.81 () {60 vars in 1483 bytes} [Mon Sep 19 11:37:13 2022] PUT /identity/v3/projects/742245b438694e51aab9349ad8364995/users/a0705f7ef17545b8aa15f3d167cae2e1/roles/753f8b35d5f0458d8f62d0ad6e28033f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:37:17.617013 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0fe36d-5d16-4241-a380-ae612a228881 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:17.617563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0fe36d-5d16-4241-a380-ae612a228881 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:17.617563 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0fe36d-5d16-4241-a380-ae612a228881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:17.660884 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-5c0fe36d-5d16-4241-a380-ae612a228881 None None] MFA Rules not processed for user `a0705f7ef17545b8aa15f3d167cae2e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:37:17.662128 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-5c0fe36d-5d16-4241-a380-ae612a228881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:37:17.803887 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1470/2921] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:37:17 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:37:17.822929 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c16e5a-76bc-4a90-950c-a38cbdac3077 admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:17.825766 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-e4c16e5a-76bc-4a90-950c-a38cbdac3077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:37:17.943071 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c16e5a-76bc-4a90-950c-a38cbdac3077 admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:17.947313 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc52cc9-a976-42af-9f28-527d664ef9e8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88670b0e1a0f460d8f74917c92db9b9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:17.947959 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc52cc9-a976-42af-9f28-527d664ef9e8 service glance] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:17.948403 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc52cc9-a976-42af-9f28-527d664ef9e8 service glance] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:17.948862 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc52cc9-a976-42af-9f28-527d664ef9e8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:18.018896 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc52cc9-a976-42af-9f28-527d664ef9e8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:18.020540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc52cc9-a976-42af-9f28-527d664ef9e8 service glance] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:18.031004 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1452/2922] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:37:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:37:18.165918 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bce082-8703-446a-9ede-2baf001717ec None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:18.166209 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bce082-8703-446a-9ede-2baf001717ec None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:18.166483 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bce082-8703-446a-9ede-2baf001717ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:18.185811 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-58bce082-8703-446a-9ede-2baf001717ec None None] MFA Rules not processed for user `06d279b488d148428b7a3c35a9786ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:37:18.187036 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-58bce082-8703-446a-9ede-2baf001717ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:37:18.201206 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:37:18.250043 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1471/2923] 10.222.0.81 () {58 vars in 1009 bytes} [Mon Sep 19 11:37:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:37:18.307080 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc52cc9-a976-42af-9f28-527d664ef9e8 service glance] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:18.308606 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-4cc52cc9-a976-42af-9f28-527d664ef9e8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:37:18.365295 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:37:18.382620 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc52cc9-a976-42af-9f28-527d664ef9e8 service glance] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:18.387716 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a100293-1c60-4ad6-ab66-0f6bfaa2a196 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8a9732807d4921a41a6a2d5019e2f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:18.388349 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a100293-1c60-4ad6-ab66-0f6bfaa2a196 service swift] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:18.388752 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a100293-1c60-4ad6-ab66-0f6bfaa2a196 service swift] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:18.389035 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a100293-1c60-4ad6-ab66-0f6bfaa2a196 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:18.397556 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a100293-1c60-4ad6-ab66-0f6bfaa2a196 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:18.399140 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a100293-1c60-4ad6-ab66-0f6bfaa2a196 service swift] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:18.408406 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1453/2924] 10.222.0.81 () {60 vars in 1384 bytes} [Mon Sep 19 11:37:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:37:28.052030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbb45d-cc37-4bc1-b977-12f0b9a7a1b5 None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:28.053070 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbb45d-cc37-4bc1-b977-12f0b9a7a1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:28.053201 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbb45d-cc37-4bc1-b977-12f0b9a7a1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:28.073013 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-c0bbb45d-cc37-4bc1-b977-12f0b9a7a1b5 None None] MFA Rules not processed for user `1d4fe64743d747ccadb431ce88fbadcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:37:28.074229 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c0bbb45d-cc37-4bc1-b977-12f0b9a7a1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:37:28.081107 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1472/2925] 10.222.0.81 () {58 vars in 958 bytes} [Mon Sep 19 11:37:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:37:28.405032 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=7 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:37:29.832051 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0d2b6-bbfa-4a93-ae1c-a920ca8d7810 None None] REQUEST_METHOD: `POST` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:29.832640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0d2b6-bbfa-4a93-ae1c-a920ca8d7810 None None] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:29.832640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0d2b6-bbfa-4a93-ae1c-a920ca8d7810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:29.851638 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.auth.core [None req-9ff0d2b6-bbfa-4a93-ae1c-a920ca8d7810 None None] MFA Rules not processed for user `06d279b488d148428b7a3c35a9786ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:37:29.852889 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9ff0d2b6-bbfa-4a93-ae1c-a920ca8d7810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:37:29.860275 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1454/2926] 10.222.0.81 () {58 vars in 1009 bytes} [Mon Sep 19 11:37:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:37:29.873390 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbb45d-cc37-4bc1-b977-12f0b9a7a1b5 None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:29.886287 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbb45d-cc37-4bc1-b977-12f0b9a7a1b5 None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:29.891436 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c5e7c-1978-4b26-b1d9-ec5f7fad343a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8a9732807d4921a41a6a2d5019e2f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:29.892187 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c5e7c-1978-4b26-b1d9-ec5f7fad343a service swift] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:29.892480 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c5e7c-1978-4b26-b1d9-ec5f7fad343a service swift] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:29.892779 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c5e7c-1978-4b26-b1d9-ec5f7fad343a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:29.900986 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c5e7c-1978-4b26-b1d9-ec5f7fad343a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:29.902587 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c5e7c-1978-4b26-b1d9-ec5f7fad343a service swift] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:29.912103 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1473/2927] 10.222.0.81 () {60 vars in 1384 bytes} [Mon Sep 19 11:37:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 11:37:39.846131 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=3 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:37:39.908173 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=26 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:37:55.642254 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0d2b6-bbfa-4a93-ae1c-a920ca8d7810 None None] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:55.643691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-9ff0d2b6-bbfa-4a93-ae1c-a920ca8d7810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:37:55.702008 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0d2b6-bbfa-4a93-ae1c-a920ca8d7810 None None] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:55.705612 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87a0f2-17ba-4971-9ead-fab988d1436e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:55.706212 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87a0f2-17ba-4971-9ead-fab988d1436e service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:55.706518 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87a0f2-17ba-4971-9ead-fab988d1436e service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:55.706793 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87a0f2-17ba-4971-9ead-fab988d1436e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:55.716561 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b87a0f2-17ba-4971-9ead-fab988d1436e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:55.717987 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b87a0f2-17ba-4971-9ead-fab988d1436e service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:55.727412 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1455/2928] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:37:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:37:57.262271 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c5e7c-1978-4b26-b1d9-ec5f7fad343a service swift] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:37:57.263848 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-279c5e7c-1978-4b26-b1d9-ec5f7fad343a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:37:57.345753 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c5e7c-1978-4b26-b1d9-ec5f7fad343a service swift] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:37:57.351282 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71308a6-fbfd-40cf-8831-a849a5c5639a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5abd70a18ccd407bafe3b61c8870986e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:37:57.351920 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71308a6-fbfd-40cf-8831-a849a5c5639a service nova] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:37:57.352200 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71308a6-fbfd-40cf-8831-a849a5c5639a service nova] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:37:57.352478 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71308a6-fbfd-40cf-8831-a849a5c5639a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:37:57.354248 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-d71308a6-fbfd-40cf-8831-a849a5c5639a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:37:57.368593 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71308a6-fbfd-40cf-8831-a849a5c5639a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:37:57.370187 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71308a6-fbfd-40cf-8831-a849a5c5639a service nova] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:37:57.383609 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1474/2929] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:37:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:38:05.723156 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=26 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:38:07.378243 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=34 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:38:33.758227 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87a0f2-17ba-4971-9ead-fab988d1436e service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:38:33.766989 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87a0f2-17ba-4971-9ead-fab988d1436e service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:38:33.770666 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e7f9c-c4bd-4e78-8342-860864a1385a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:38:33.771251 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e7f9c-c4bd-4e78-8342-860864a1385a service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:38:33.771532 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e7f9c-c4bd-4e78-8342-860864a1385a service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:38:33.771840 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e7f9c-c4bd-4e78-8342-860864a1385a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:38:33.773542 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-546e7f9c-c4bd-4e78-8342-860864a1385a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:38:33.787239 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546e7f9c-c4bd-4e78-8342-860864a1385a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:38:33.788683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546e7f9c-c4bd-4e78-8342-860864a1385a service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:38:33.799529 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1456/2930] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:38:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:38:34.990275 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71308a6-fbfd-40cf-8831-a849a5c5639a service nova] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:38:34.999660 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71308a6-fbfd-40cf-8831-a849a5c5639a service nova] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:38:35.003272 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a14390-d04c-4f7b-ac75-747806c9301d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:38:35.003936 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a14390-d04c-4f7b-ac75-747806c9301d service neutron] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:38:35.004746 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a14390-d04c-4f7b-ac75-747806c9301d service neutron] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:38:35.005018 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a14390-d04c-4f7b-ac75-747806c9301d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:38:35.006726 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c3a14390-d04c-4f7b-ac75-747806c9301d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:38:35.019845 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a14390-d04c-4f7b-ac75-747806c9301d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:38:35.021236 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a14390-d04c-4f7b-ac75-747806c9301d service neutron] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:38:35.032031 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1475/2931] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:38:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:38:43.794683 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=6 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:38:45.027117 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=6 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:39:10.859334 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e7f9c-c4bd-4e78-8342-860864a1385a service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:39:10.868173 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e7f9c-c4bd-4e78-8342-860864a1385a service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:39:10.871921 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5180c41-7a92-41c4-bd69-d7d332a44b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65de2e5f67dd4bb6a20cf3934afc8abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:39:10.872743 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5180c41-7a92-41c4-bd69-d7d332a44b98 service neutron] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:39:10.873019 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5180c41-7a92-41c4-bd69-d7d332a44b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:39:10.873286 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5180c41-7a92-41c4-bd69-d7d332a44b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:39:10.875064 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-f5180c41-7a92-41c4-bd69-d7d332a44b98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:39:10.886058 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5180c41-7a92-41c4-bd69-d7d332a44b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:39:10.887486 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5180c41-7a92-41c4-bd69-d7d332a44b98 service neutron] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:39:10.901400 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1457/2932] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:39:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:39:18.212758 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a020eb-a77d-46bd-8657-02450a1754ed None None] REQUEST_METHOD: `POST` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:39:18.213238 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a020eb-a77d-46bd-8657-02450a1754ed None None] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:39:18.213353 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a020eb-a77d-46bd-8657-02450a1754ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:39:18.232163 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.auth.core [None req-b7a020eb-a77d-46bd-8657-02450a1754ed None None] MFA Rules not processed for user `06d279b488d148428b7a3c35a9786ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81461) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 11:39:18.233351 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b7a020eb-a77d-46bd-8657-02450a1754ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:39:18.240514 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1476/2933] 10.222.0.81 () {58 vars in 967 bytes} [Mon Sep 19 11:39:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 11:39:18.280800 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5180c41-7a92-41c4-bd69-d7d332a44b98 service neutron] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:39:18.293123 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5180c41-7a92-41c4-bd69-d7d332a44b98 service neutron] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:39:18.298219 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff9ff3-2c9a-48b3-81e8-f1cd7a2b1d67 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8a9732807d4921a41a6a2d5019e2f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:39:18.298819 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff9ff3-2c9a-48b3-81e8-f1cd7a2b1d67 service swift] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:39:18.299100 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff9ff3-2c9a-48b3-81e8-f1cd7a2b1d67 service swift] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:39:18.299372 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff9ff3-2c9a-48b3-81e8-f1cd7a2b1d67 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:39:18.307517 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff9ff3-2c9a-48b3-81e8-f1cd7a2b1d67 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:39:18.309401 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff9ff3-2c9a-48b3-81e8-f1cd7a2b1d67 service swift] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:39:18.317730 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1458/2934] 10.222.0.81 () {60 vars in 1384 bytes} [Mon Sep 19 11:39:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 11:39:21.449474 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a020eb-a77d-46bd-8657-02450a1754ed None None] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:39:21.451073 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-b7a020eb-a77d-46bd-8657-02450a1754ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:39:21.562942 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a020eb-a77d-46bd-8657-02450a1754ed None None] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:39:21.568016 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b373ba1-7a27-48d2-bb98-4501d9485ad2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e7a18dc4a4d4d35a05f1aafbaaf9c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:39:21.568621 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b373ba1-7a27-48d2-bb98-4501d9485ad2 service placement] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:39:21.568901 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b373ba1-7a27-48d2-bb98-4501d9485ad2 service placement] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:39:21.569172 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b373ba1-7a27-48d2-bb98-4501d9485ad2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:39:21.570922 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-7b373ba1-7a27-48d2-bb98-4501d9485ad2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:39:21.583778 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b373ba1-7a27-48d2-bb98-4501d9485ad2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:39:21.585465 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b373ba1-7a27-48d2-bb98-4501d9485ad2 service placement] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:39:21.595957 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1477/2935] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:39:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:39:24.551793 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff9ff3-2c9a-48b3-81e8-f1cd7a2b1d67 service swift] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:39:24.564599 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff9ff3-2c9a-48b3-81e8-f1cd7a2b1d67 service swift] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:39:24.570085 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c9f0a-857e-4dfc-874d-34d911a37aa0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e7a18dc4a4d4d35a05f1aafbaaf9c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:39:24.571564 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c9f0a-857e-4dfc-874d-34d911a37aa0 service placement] REQUEST_METHOD: `GET` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:39:24.572114 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c9f0a-857e-4dfc-874d-34d911a37aa0 service placement] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:39:24.572563 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c9f0a-857e-4dfc-874d-34d911a37aa0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:39:24.575888 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-760c9f0a-857e-4dfc-874d-34d911a37aa0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:39:24.592613 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c9f0a-857e-4dfc-874d-34d911a37aa0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:39:24.594674 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c9f0a-857e-4dfc-874d-34d911a37aa0 service placement] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:39:24.607859 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b373ba1-7a27-48d2-bb98-4501d9485ad2 service placement] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:39:24.621399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1459/2936] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:39:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:39:24.689747 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b373ba1-7a27-48d2-bb98-4501d9485ad2 service placement] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:39:24.695037 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c735836-0fd8-474b-8a4c-86a4a941c4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:39:24.695671 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c735836-0fd8-474b-8a4c-86a4a941c4c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:39:24.695979 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c735836-0fd8-474b-8a4c-86a4a941c4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:39:24.696257 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c735836-0fd8-474b-8a4c-86a4a941c4c4 admin admin] PATH_INFO: `/v3/users/1d4fe64743d747ccadb431ce88fbadcb` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:39:24.697420 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c735836-0fd8-474b-8a4c-86a4a941c4c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d4fe64743d747ccadb431ce88fbadcb)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:39:24.699323 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c735836-0fd8-474b-8a4c-86a4a941c4c4 admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:39:24.745207 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4c735836-0fd8-474b-8a4c-86a4a941c4c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d4fe64743d747ccadb431ce88fbadcb'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:39:24.745900 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4c735836-0fd8-474b-8a4c-86a4a941c4c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d4fe64743d747ccadb431ce88fbadcb'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:39:24.746506 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4c735836-0fd8-474b-8a4c-86a4a941c4c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d4fe64743d747ccadb431ce88fbadcb'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:39:24.753331 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4c735836-0fd8-474b-8a4c-86a4a941c4c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d4fe64743d747ccadb431ce88fbadcb'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:39:24.764859 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-4c735836-0fd8-474b-8a4c-86a4a941c4c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d4fe64743d747ccadb431ce88fbadcb'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:39:24.777438 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1478/2937] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:39:24 2022] DELETE /identity/v3/users/1d4fe64743d747ccadb431ce88fbadcb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 11:39:24.948092 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c9f0a-857e-4dfc-874d-34d911a37aa0 service placement] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:39:24.963801 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c9f0a-857e-4dfc-874d-34d911a37aa0 service placement] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:39:25.016882 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebdbac-2725-4f6d-bbec-5d050f6c763a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:39:25.017509 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebdbac-2725-4f6d-bbec-5d050f6c763a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:39:25.017790 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebdbac-2725-4f6d-bbec-5d050f6c763a admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:39:25.018067 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebdbac-2725-4f6d-bbec-5d050f6c763a admin admin] PATH_INFO: `/v3/projects/67f0a300a82b480d8f4a2c0a19f94fb6` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:39:25.019714 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebdbac-2725-4f6d-bbec-5d050f6c763a admin admin] RBAC: Authorizing `identity:delete_project(project_id=67f0a300a82b480d8f4a2c0a19f94fb6)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:39:25.021658 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebdbac-2725-4f6d-bbec-5d050f6c763a admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:39:25.066682 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bbebdbac-2725-4f6d-bbec-5d050f6c763a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67f0a300a82b480d8f4a2c0a19f94fb6', 'request_id': 'req-bbebdbac-2725-4f6d-bbec-5d050f6c763a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:39:25.072977 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bbebdbac-2725-4f6d-bbec-5d050f6c763a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67f0a300a82b480d8f4a2c0a19f94fb6', 'request_id': 'req-bbebdbac-2725-4f6d-bbec-5d050f6c763a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:39:25.078576 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bbebdbac-2725-4f6d-bbec-5d050f6c763a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67f0a300a82b480d8f4a2c0a19f94fb6', 'request_id': 'req-bbebdbac-2725-4f6d-bbec-5d050f6c763a'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:39:25.091744 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bbebdbac-2725-4f6d-bbec-5d050f6c763a admin admin] The token cache is being invalidate because project 67f0a300a82b480d8f4a2c0a19f94fb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:39:25.092126 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bbebdbac-2725-4f6d-bbec-5d050f6c763a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:39:25.092640 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-bbebdbac-2725-4f6d-bbec-5d050f6c763a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:39:25.094103 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1460/2938] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:39:24 2022] DELETE /identity/v3/projects/67f0a300a82b480d8f4a2c0a19f94fb6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:39:25.098759 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c735836-0fd8-474b-8a4c-86a4a941c4c4 admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:39:25.100246 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-4c735836-0fd8-474b-8a4c-86a4a941c4c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:39:25.108585 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:39:25.114232 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c735836-0fd8-474b-8a4c-86a4a941c4c4 admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:39:25.171487 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8da0b-396b-48b2-95e4-543ef80c255b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:39:25.172140 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8da0b-396b-48b2-95e4-543ef80c255b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:39:25.172423 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8da0b-396b-48b2-95e4-543ef80c255b admin admin] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:39:25.172694 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8da0b-396b-48b2-95e4-543ef80c255b admin admin] PATH_INFO: `/v3/users/a0705f7ef17545b8aa15f3d167cae2e1` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:39:25.173819 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8da0b-396b-48b2-95e4-543ef80c255b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0705f7ef17545b8aa15f3d167cae2e1)` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:39:25.175750 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8da0b-396b-48b2-95e4-543ef80c255b admin admin] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:39:25.223074 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c9b8da0b-396b-48b2-95e4-543ef80c255b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0705f7ef17545b8aa15f3d167cae2e1'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:39:25.223723 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c9b8da0b-396b-48b2-95e4-543ef80c255b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0705f7ef17545b8aa15f3d167cae2e1'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:39:25.224367 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c9b8da0b-396b-48b2-95e4-543ef80c255b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0705f7ef17545b8aa15f3d167cae2e1'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:39:25.231276 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c9b8da0b-396b-48b2-95e4-543ef80c255b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0705f7ef17545b8aa15f3d167cae2e1'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:39:25.243089 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.notifications [None req-c9b8da0b-396b-48b2-95e4-543ef80c255b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0705f7ef17545b8aa15f3d167cae2e1'} {{(pid=81461) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:39:25.255485 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1479/2939] 10.222.0.81 () {58 vars in 1224 bytes} [Mon Sep 19 11:39:25 2022] DELETE /identity/v3/users/a0705f7ef17545b8aa15f3d167cae2e1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:39:25.423887 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebdbac-2725-4f6d-bbec-5d050f6c763a admin admin] Authenticating user token {{(pid=81460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:39:25.425918 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.fernet_utils [None req-bbebdbac-2725-4f6d-bbec-5d050f6c763a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:39:25.440070 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebdbac-2725-4f6d-bbec-5d050f6c763a admin admin] Validating token access rules against request {{(pid=81460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:39:25.491829 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:39:25.504189 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff08ea9-bfc5-445a-be63-40be5e6ab13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd316848081f4ae2ab8647785189b3f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d5c1387f77e4df7ac9b4d83d404bc68', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:39:25.504842 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff08ea9-bfc5-445a-be63-40be5e6ab13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:39:25.505123 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff08ea9-bfc5-445a-be63-40be5e6ab13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:39:25.505399 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff08ea9-bfc5-445a-be63-40be5e6ab13f admin admin] PATH_INFO: `/v3/projects/742245b438694e51aab9349ad8364995` {{(pid=81460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:39:25.506511 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff08ea9-bfc5-445a-be63-40be5e6ab13f admin admin] RBAC: Authorizing `identity:delete_project(project_id=742245b438694e51aab9349ad8364995)` {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:39:25.508492 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff08ea9-bfc5-445a-be63-40be5e6ab13f admin admin] RBAC: Authorization granted {{(pid=81460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:39:25.553073 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-aff08ea9-bfc5-445a-be63-40be5e6ab13f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '742245b438694e51aab9349ad8364995', 'request_id': 'req-aff08ea9-bfc5-445a-be63-40be5e6ab13f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:39:25.559540 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-aff08ea9-bfc5-445a-be63-40be5e6ab13f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '742245b438694e51aab9349ad8364995', 'request_id': 'req-aff08ea9-bfc5-445a-be63-40be5e6ab13f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:39:25.565691 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-aff08ea9-bfc5-445a-be63-40be5e6ab13f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '742245b438694e51aab9349ad8364995', 'request_id': 'req-aff08ea9-bfc5-445a-be63-40be5e6ab13f'} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:39:25.577819 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-aff08ea9-bfc5-445a-be63-40be5e6ab13f admin admin] The token cache is being invalidate because project 742245b438694e51aab9349ad8364995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 11:39:25.578109 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-aff08ea9-bfc5-445a-be63-40be5e6ab13f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:39:25.578770 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG keystone.notifications [None req-aff08ea9-bfc5-445a-be63-40be5e6ab13f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 11:39:25.580270 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: [pid: 81460|app: 0|req: 1461/2940] 10.222.0.81 () {58 vars in 1233 bytes} [Mon Sep 19 11:39:25 2022] DELETE /identity/v3/projects/742245b438694e51aab9349ad8364995 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 11:39:35.251599 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=52,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:39:35.574823 dv-cinder-820994-5-iscsi devstack@keystone.service[81460]: DEBUG dbcounter [-] [81460] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 11:41:46.071528 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8da0b-396b-48b2-95e4-543ef80c255b admin admin] Authenticating user token {{(pid=81461) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 11:41:46.072977 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-c9b8da0b-396b-48b2-95e4-543ef80c255b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:41:46.210331 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8da0b-396b-48b2-95e4-543ef80c255b admin admin] Validating token access rules against request {{(pid=81461) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 11:41:46.267405 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3018708-c7ad-4d40-bbfd-c9b4ec5dc86a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e7a18dc4a4d4d35a05f1aafbaaf9c31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed9a0b626e3e4b93b391158e981ec586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81461) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 11:41:46.268030 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3018708-c7ad-4d40-bbfd-c9b4ec5dc86a service placement] REQUEST_METHOD: `GET` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 11:41:46.268313 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3018708-c7ad-4d40-bbfd-c9b4ec5dc86a service placement] SCRIPT_NAME: `/identity` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 11:41:46.268585 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3018708-c7ad-4d40-bbfd-c9b4ec5dc86a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81461) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 11:41:46.270316 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.fernet_utils [None req-d3018708-c7ad-4d40-bbfd-c9b4ec5dc86a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81461) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 11:41:46.281871 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3018708-c7ad-4d40-bbfd-c9b4ec5dc86a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 11:41:46.283425 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3018708-c7ad-4d40-bbfd-c9b4ec5dc86a service placement] RBAC: Authorization granted {{(pid=81461) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 11:41:46.293873 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: [pid: 81461|app: 0|req: 1480/2941] 10.222.0.81 () {60 vars in 1365 bytes} [Mon Sep 19 11:41:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 11:41:56.289618 dv-cinder-820994-5-iscsi devstack@keystone.service[81461]: DEBUG dbcounter [-] [81461] Writing DB stats keystone:SELECT=89 {{(pid=81461) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}